Advanced Sustainable Engineering Vol. 01 (2021) No.

01 10-16

Advanced Sustainable Engineering

https://ukischolarsnetwork.org/index.php/ase

Object Detection Based on Clustering for Assistive Technology

Dwi Maryono1*, Andreas Wegiq Adia Hendix 1, Subagya2, Sukarmin3, Dewanto Harjunowibowo3*
1
Department of Informatics Education Program, Universitas Sebelas Maret, Ir Sutami 36A, Surakarta, 57126, Indonesia
2
Department of Special Education, Universitas Sebelas Maret, Ir Sutami 36A, Surakarta, 57126, Indonesia
3
Department of Physics Education, Universitas Sebelas Maret, Ir Sutami 36A, Surakarta, 57126, Indonesia
Email: dwimaryono@staff.uns.ac.id

Article history: ABSTRACT

Received : 13 Dec 2020
Revised : 24 Jun 2021 Lidar is a well-known sensor in autonomous vehicles because of its capability in object
Accepted : 26 Aug 2021 detection. However, utilise the sensor in assistive technology for blind and impaired vision
person are rare to be found. In this study, an assistive technology (AT) was made utilising Lidar
Keywords: sensor and Clustering algorithm. K-means algorithm and Kalman filter were used to improve
Assistive Technology the AT performance. Moreover, the AT is equipped with voice mode to inform the user when
Blind people the closest object was detected. Furthermore, the application of a monitoring system was built
Clustering algorithm based on the Internet of things (IoT). The application installed in android smartphones allows
Impaired vision
Internet of things
the installer to monitor the AT user and recognise the user's objects remotely. The results show
Lidar that the AT could accurately detect an object and send the information to the monitoring
application system via internet. The AT system is potentially used to help a blind person to
mobile safely.

map [6], hence mapping the surrounding better.

I. INTRODUCTION This technology was applied to a robot, unmanned
Mordini et al. argued that assistive technology vehicles, better GPS alternative, topology contour
(AT) for blind people has more attention than of an area, and obstacle avoidance [7]. Moreover,
deafness and autism spectrum disorder [1]. Some compared to other localisation techniques such as
ATs for blindness and visual impairment are haptic Absolute positioning sensors, Odometry, and GPS-
aids, traveling aids, accessible information and IMU fusion, SLAM has robustness and accuracy
communication aids, daily living [2], and [8].
smartphone applications. All of these ATs are Many researchers have made applications of
concerning on improvement of blind people's life SLAM as object avoidance. Joshi et al. [9] using the
quality. However, accessibility and economic cost SLAM algorithm in a smartphone and its inbuilt
become the main challenges [3], [4]. Therefore, sensors to mapping the outdoor environment and
technological advances for enriching online GPS as the navigation for blind people. The system
communication will affect the AT future trends. will vibrate to alert the user when obstacles are
One of the technologies potentially used to help detected. The system could detect many obstacles,
blind people "see" their environment is also staircase and alert the user. However, the
Simultaneous Localization and Mapping (SLAM). system could not tell the user via audio mode.
SLAM is defined as a mapping process on the Yagfarov et al. [10] used Light Detection and
unidentified environment, and user position is also Ranging (Lidar) as a sensor to construct a 2D map
measured simultaneously [5]. The SLAM visual on a robot and compare its accuracy against a map
currently uses sonar and laser sensors to build a 2D presented by a FARO laser tracker in a static indoor

10 UKIScholarsnetwork | ISSN: xxxx-xxxx

environment. The Lidar was built with many in each dataset. This number is either predefined
libraries such as Google Cartographer, Gmapping, manually by users for each dataset based on the
and Hector SLAM. As a result, Google dataset and each study's requirement or
Cartographer SLAM has the best accuracy against automatically defined based on some techniques,
FARO laser tracker with micrometer scale accuracy such as the elbow method or the Silhouette analysis
for all scenarios. method [16] after analysing the dataset.
Moreover, a SLAM-based semantic indoor The data learning process is shown in Fig. 1; a
navigation system was built by Zhang et al. [11] on suitable K must be determined initially at the
a robot. The system integrated multiple sensors beginning. After defining K clusters, new centroids
such as an RGB-D camera, inertial measurement are selected. The K-means algorithm randomly
unit (IMU), and the camera while the user's selects K centroids, 𝑣1 , 𝑣2 , … 𝑣𝐾 , which are used as
localisation and trajectory were achieved using a the initial center points for every cluster in the
particle filter. The localisation and semantic digital dataset. Since this step is performed randomly,
map used to navigate the user to the desired room. those points can either be the same data in the
Thanks to the audio feature, the system can help the dataset or different ones.
user conveniently and easily. However, this system In the second step, K groups are created in the
has not been tested outdoor, which has a different dataset based on randomly selected centroids.
condition and distraction. Besides, multiple object Subsequently, the distances of all the points in the
detection was investigated by Nguyen et al. [12] dataset to those K center points are calculated, and
based on clustering and deep learning methods. each data point is grouped into the cluster of each
They used Lidar as the sensor and K-Means centroid with the smallest distance. This calculation
clustering combined with density-based spatial is performed using Equation (1)[17], where 𝐶𝑗 is the
clustering of application with noise algorithms to jth cluster, ‖𝑥𝑖 − 𝑣𝑗 ‖ is the Euclidean distance
cluster meaningful data and improve the between 𝑥𝑖 and 𝑣𝑗 , and 𝑣𝑗 is centroid of cluster 𝐶𝑗 .
performance.
By comparing these distances, all the points in the
Therefore, to assist the visually impaired for
data are defined to their cluster. After calculating all
mobility, an AT should consider its efficiency,
the data points in the dataset, K groups are formed
accuracy, and economic affordability. Hence, in this
with the respective member points.
report, a combination of Lidar and K-Means
The third step involves calculating each group's
clustering with voice mode [13] would be useful
mean by Equation (2) [17] to obtain the new centers
and low-cost for mass production. The Lidar sensor
for All K groups. These new values typically differ
is used to detect and measure the distance from the
from the initially selected ones. Subsequently, the
user to the transparent objects (glass, water) [14],
algorithms will repeat the second step to define the
[15]. However, since the Lidar working principle is
new K group depending on the new centroids and
based on light, too much dust in the air like fog and
the distance from all the data points to those center
snow outside the room will distract the function [8],
points. The third step is repeated to obtain the new
causing inaccuracy then must be in consideration. A
centroids for the K groups of data points in the last
Global Positioning System (GPS) will allow
step. K-means clustering will stop when no changes
families member to have monitoring access to the
occur in the new centroids, which means that the
users for safety reasons.
previous clustering values are the same as those of
II. METHODS the new ones. At this step, the algorithm has finally
clustered all data points into K different groups of
A. Algorithms implemented data points with the same characteristics.
The K-means clustering algorithm is a method of
vector quantisation that aims to separate some 𝐶𝑗 = {𝑥𝑖 : ‖𝑥𝑖 − 𝑣𝑗 ‖2 ≤ ‖𝑥𝑖 − 𝑣𝑚 ‖2 , ∀𝑖, 1 ≤ 𝑚 ≤ 𝐾} (1)
initial input data into K clusters, in which each data for 1 ≤ 𝑗 ≤ 𝐾.
point belongs to their specific group with the 1
closest mean of that group. This algorithm is a 𝑣𝑗 = |𝐶 | ∑𝑥𝑖 ∈𝐶𝑗 𝑥𝑖 (2)
𝑗
simple yet popular and useful unsupervised
machine learning algorithm. The letter "K" in the
algorithm's name represents the number of clusters

11 Maryono et al./Advanced Sustainable Engineering Vol. 01 (2021) No. 01 10-16

 subscribed topic of /viz which contains position
data of the six objects with their x, y, and z
coordinates. The third data were used to determined
objects in front of the user. To determine the closest
object, the system must calculate the closest
distance by using a cosinus formula. To determine
the object degree, the system used an arctan
calculation. Based on the calculation, the system
chose the recorded voices. Simultaneously, there
was a HTTP POST request to send the six object
clusters data to the online database. The algorithm
was keep iterated until a termination signal received
from the user.

Fig. 1. K-means clustering algorithm: (a) Initial input data; (b) Randomly
select centroids; (c) Cluster input data into K clusters; (d) Obtain new
centroids

B. Architecture system
The system works utilising a robotic operation
system (ROS) platform. The algorithms followed
Fig. 2. The RPLidar package collected the raw data
from the sensor and converted it into LaserScan
data (sensor_msgs/LaserScan.msg). The LaserScan
data published to ROS topic, namely topic /scan
and subscribed by vigir_laserscan_to_pointcloud
package and converted into PointCloud2
(sensor_msgs/PointCloud2.msg). The conversion Fig. 2. Architecture system based on clustering algorithms
was needed by multiple-object-tracking-lidar
package to make clusterisation using clustering III. EXPERIMENT RESULT
algorithm and Kalman filter algorithm. After This study was conducted to design assistive
transformation, the PointCloud2 data published to technology for a blind and impaired vision people.
ROS topic, namely topic /filtered_cloud. A. Assistive Technology
The multiple-object-tracking-lidar package
subscribed the topic /filtered_cloud to import the In this study, the helmet technology assistive for
filtered PointCloud2 data. They were used for blind people, and impaired vision had resulted. Fig.
clusterisation of the objects to six objects with 3 shows the Lidar sensor was installed on the top
PointCloud2 data type. Each objects then published with the Jetson nano as the processor unit. The
into topic of /cluster_0, /cluster_1, /cluster_2, headset were taken off and altered with speaker for
/cluster_3, /cluster_4, and /cluster_5. Afterward, the testing. The system was powered by an 5V 22Ah
system got the x, y, and z positions by calculated power bank. The weight were measured and found
the mean value of the points in the PointCloud2 to be 700grams.
cluster. The cluster position with x, y, and z
coordinate pushed into a topic, namely /viz with
MarkerArray data type
(visualization_msgs/MarkerArray.msg). The RVIZ
programme ran to visualised the six PointCloud2
clusters data, MarkerArray data, LaserScan data,
and filtered PointCloud2 data. Fig. 3. Technology assistive helmet appearance
Besides that, the talker node ran an algorithm to
B. Testing
produced voice alerts to the user. The node

12 Maryono et al./Advanced Sustainable Engineering Vol. 01 (2021) No. 01 10-16

The Lidar could detect two objects, as shown in Fig.
4. The system shows the performance in object
clusterisation. It detects the wall but the two objects
closed by. The mapping could be seen in Fig. 5. It
is obviously can be seen that the Lidar could not
detect the very close object around 10 centimetres.
The testing was conducted to detect objects bit far
from the sensor and works perfectly, as shown in

Fig. 6.

Fig. 6. 2D scanning result in objects in a room

C. Measurement Calibration
The Lidar used was tested to measure the
distance from the Lidar to a standing board using a
meter. The Lidar was placed in front of the board at
Fig. 4. RPLidar A2 in detection testing
150 cm, as shown in Fig. 7. It showed the distance
measurement found to be (152±1)cm. Therefore,
the accuracy was 99.9%, which is remarkable. For
assistive technology, this accuracy is more than
enough.

Fig. 5. 2D scanning result

13 Maryono et al./Advanced Sustainable Engineering Vol. 01 (2021) No. 01 10-16

 Fig. 7. Measurement testing experimental

D. Monitoring system application

An application based on android was used to
detect the user position by using the Internet. There
were three main features named location tracker,
manual user, and about the system, as shown in Fig.
8. The application was also receiving the voice data
from the assistive technology in real-time.
Therefore it helps the family to monitor the user
and knowing the objects surrounding it.

14 Maryono et al./Advanced Sustainable Engineering Vol. 01 (2021) No. 01 10-16

 Fig. 8. The GPS IoT based

IV. CONCLUSIONS
This research aimed to investigate assistive
technology's performance based on Lidar sensor
and K-Means clustering algorithm on object
detection for blind and impaired vision people. The
results show that the product is potentially used as
an assistive technology since it detects the user's
objects. Moreover, it has a monitoring system IoT
based; hence the family knows the user location and
the objects faced by him/her.
ACKNOWLEDGEMENT
The authors thank the Ministry of Education and
Culture of Indonesia for grant no
02/E2/PPK/SPK/PENSUS/2020.
REFERENCES
[1] E. Mordini et al., Assistive Technologies for People
with Disabilities. Part II: Current and emerging
technologies, vol. 2, no. January. 2018.
[2] M. A. Hersh, “Perception, the Eye and Assistive
Technology Issues BT - Assistive Technology for
Visually Impaired and Blind People,” M. A. Hersh
and M. A. Johnson, Eds. London: Springer London,
2008, pp. 51–101.
[3] R. O. Smith et al., “Disability and Rehabilitation :
Assistive Technology Assistive technology products :
a position paper from the first global research ,
innovation , and education on assistive technology
( GREAT ) summit,” Disabil. Rehabil. Assist.
Technol., vol. 13, no. 5, pp. 473–485, 2018.
[4] S. S. Senjam, A. Foster, and C. Bascaran, “Disability
and Rehabilitation : Assistive Technology Barriers to
using assistive technology among students with visual
disability in schools for the blind in Delhi , India,”

15 Maryono et al./Advanced Sustainable Engineering Vol. 01 (2021) No. 01 10-16

 Disabil. Rehabil. Assist. Technol., vol. 0, no. 0, pp. 1–
5, 2020.
[5] B. Y. T. I. M. Bailey and H. Durrant-whyte,
“Simultaneous Localization and Mapping ( SLAM ):,”
IEEE Robot. Autom. Mag., vol. 13, no. September, pp.
108–117, 2006.
[6] J. Fuentes-Pacheco, J. Ruiz-Ascencio, and J. M.
Rendón-Mancha, “Visual simultaneous localization
and mapping: a survey,” Artif. Intell. Rev., vol. 43, no.
1, pp. 55–81, 2015.
[7] C. Cadena et al., “Past , Present , and Future of
Simultaneous Localization and Mapping : Toward the
Robust-Perception Age,” IEEE Trans. Robot., vol. 32,
no. 6, pp. 1309–1332, 2016.
[8] E. Yurtsever, J. Lambert, A. Carballo, K. Takeda, and
S. Member, “A Survey of Autonomous Driving :
Common Practices and Emerging Technologies,” vol.
8, 2020.
[9] A. Joshi, H. Agrawal, and P. Agrawal, “Simultaneous
Localization and Mapping for Visually Impaired
People for Outdoor Environment,” in Proceedings of
the Second International Conference on Computer
and Communication Technologies, Advances in
Intelligent Systems and Computing 381, 2016, pp.
107–115.
[10] R. Yagfarov, M. Ivanou, and I. Afanasyev, “Map
Comparison of Lidar-based 2D SLAM Algorithms
Using Precise Ground Truth,” 2018 15th Int. Conf.
Control. Autom. Robot. Vision, ICARCV 2018, no.
November, pp. 1979–1983, 2018.
[11] X. Zhang, B. Li, S. L. Joseph, J. P. Muñoz, and C. Yi,
“A SLAM based Semantic Indoor Navigation System
for Visually Impaired Users,” in 2015 IEEE
International Conference on Systems, Man, and
Cybernetics A, 2015, pp. 1458–1463.
[12] H. T. Nguyen, E. H. Lee, C. H. Bae, and S. Lee,
“Multiple object detection based on clustering and
deep learning methods,” Sensors (Switzerland), vol.
20, no. 16, pp. 1–14, 2020.
[13] S. S. Senjam, A. Foster, C. Bascaran, and P. Vashist,
“Assistive Technology Assistive technology for
students with visual disability in schools for the blind
in Delhi Assistive technology for students with visual
disability in schools for the blind,” Disabil. Rehabil.
Assist. Technol., vol. 0, no. 0, pp. 1–7, 2019.
[14] P. Foster, Z. Sun, J. J. Park, and B. Kuipersl,
“VisAGGE : Visible Angle Grid for Glass
Environments,” pp. 2213–2220, 2013.
[15] S. Morsy, A. Shaker, and A. El-Rabbany, “Using
Multispectral Airborne LiDAR Data for Land/Water
Discrimination : A Case Study at Lake Ontario,
Canada,” Appl. Sci., vol. 8, no. 349, pp. 1–21, 2018.
[16] P. J. Rousseeuw, “Silhouettes : a graphical aid to the
interpretation and validation of cluster analysis,” vol.
20, pp. 53–65, 1987.
[17] P. Palanisamy, “praveen-palanisamy/multiple-object-
tracking-lidar: Multiple-Object-Tracking-from-Point-
Clouds_v1.0.2,” Dec. 2019.

16 Maryono et al./Advanced Sustainable Engineering Vol. 01 (2021) No. 01 10-16