See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/352167430

Manhole cover monitoring system over IOT

Article · January 2021

CITATIONS READS
0 367

3 authors, including:

Wesam moneer Rasheed Raed Abdulla

Asia Pacific University of Technology and Innovation Asia Pacific University of Technology and Innovation
1 PUBLICATION   0 CITATIONS    55 PUBLICATIONS   80 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

RFID based system for IoT View project

All content following this page was uploaded by Raed Abdulla on 06 June 2021.

The user has requested enhancement of the downloaded file.

 Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 3, (2021) 1

Manhole cover monitoring system over IOT

Wesam Moneer Rasheed Raed Abdulla Low Yee San
School of Engineering School of Engineering School of Engineering
Asia Pacific University of Technology Asia Pacific University of Technology Asia Pacific University of Technology
& Innovation (APU) & Innovation (APU) & Innovation (APU)
Kuala Lumpur, Malaysia Kuala Lumpur, Malaysia Kuala Lumpur, Malaysia
TP046026@mail.apu.edu.my raed@staffemail.apu.edu.my yee.san@staffemail.apu.edu.my

Abstract— Manhole recognition and monitoring is one of Development of smart cities and implementation of
the essential needs for modern society, particularly smart city automated manhole covers has gained much importance in
plan. The idea of this project roots in fact that missing or stolen previous years. A number of incidents occur regarding the
manholes results in various road accidents and it shrinks the safety and security of people due to the issues of manhole
quality of city. The major challenge in this research is to cover (MC). Uncovered holes pose a great challenge to
investigate a method in recognition of manholes and further citizens and can cause safety hazards to underground
investigation on the condition of manhole on road. The report structures. To prevent accidents from manhole-cover an
can be simultaneously updated in IOT platform which can be intelligent manhole cover management system has been
further used with other applications such as WAZE or Google
introduced as a basic platform in smart cities [7].
MAP. This method uses Naïve Bayes classifier to train dataset
and detect as well as recognize the missing manhole on road. Developing cities usually do not focus on opened manhole
This project has three main components including 1) Recognize covers and are not monitored properly. These manholes can
manhole on road, 2) Detect missing manhole, and 3) Update the be a great threat to lives and assets in a number of ways.
coordinates to IOT platform. The software used for image Manholes can be filled with some toxic and hazardous
processing was MATLAB. The coordinates were obtained via materials and underground structures can get affected from
Arduino Mega and SIM808 GSM modem to updates the missing these materials. Systems need to be developed to monitor the
manhole covers coordinates on BLYNK IOT. The result
lids of manhole covers to avoid accidents. Developments are
obtained from the accuracy of the Naïve Bayes classifier
indicates that the system can perfectly detect the missing
made to replace traditional and manual methods with IoT
manhole with total accuracy of 85%. The True Negative Rate based automated monitoring systems [8].
(TNR), Positive Predictive Value (PPV) and False Discovery The issue of poorly managed and stolen gas well covers is
Rate (FDR) were found to be 80%, 90%, and 10%, respectively. becoming an alarming situation in different countries. The
The data shows that the system results in 100% accuracy in existing manhole cover systems are found to be covering
detection for speed of below 20 KMPH. The result indicates that single monitoring parameters, have immature technology and
the camera can varied from 30 to 42 degrees while obtaining best contain inefficient analysis capabilities to find and eradicate
result without any misdetection. The rate of recognition for the
issues regarding manhole covers and security. The traditional
system with 30 fps is found to be 71.58% by dividing the total
time for processing a frame over the total frame time duration).
methods of manhole cover protection and monitoring cannot
The Latency test in obtaining the coordinates indicates that the cope with the challenges of increasing population and
proposed system takes 4.724 seconds in average to obtain the underground infrastructures, therefore, there is a need to
coordinates from GSM modem. develop more automatic systems of monitoring [9].
To detect the issues regarding manholes, a system with
Keywords— Manhole cover, microcontroller, google Global Positioning system (GPS) has been developed based
map, smart city application, arduino, Internet of Things. on ultrasonic sensors and accelerometers to detect manholes
accurately. For monitoring of the road, the sensor-based
I. INTRODUCTION system detects the vehicles and provides data to cloud system
continuously. This system helps drivers to avoid manhole on
Every From a couple of years, the failure of Manhole
the road and provide them with alarming alerts. The sensor-
Cover (MC) is gaining more importance than ever. Failure of
based system also provides useful data to Highway
MC can have severe impacts on economy, security and safety
Maintenance Department in a smart city. This proposed
of a region. Traditional methods of control measures cannot
method employs accelerometer and ultrasonic sensors. The
fill the void that has been created from a number of incidents
data obtained from sensors is processed and analysed through
in city areas. Therefore, there is a need of full automated
Honey Bee optimization algorithms. This system can also
monitoring systems and it has now become the part of smart
provide information of any accident occurred on the road to
cities development [1].
save other vehicles from its effects. A survey report showed
In most of the smart city applications, Internet of Things that in India, nearly 50% of the roads are damaged because of
(IoT) devices are being employed for automatic monitoring manholes and can cause dangerous impacts on humans and
systems [2-5]. Monitoring of manhole cover is gaining more vehicles. The sensor-based systems are seen to be more fast
importance in IoT applications. Issues relating to manhole and liable modes of manhole detection [10].
cover affects the security, safety and economy of a society. A
Another method of manhole detection is Light Detection
number of challenges rise, as there are just 30% MC
and Ranging (LiDAR) which works on infrared rays for
monitoring systems based on automatic structures, which need
detection. A 2D and 3D LiDAR system can be developed
to be extended from circuit design point of view [6].
based on camera detection to detect manholes. This system is
 Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 3, (2021) 2

connected to a computer imaging program to detect manholes. and [13] , have tested this system with a speed of 2KMPH for
Detecting systems can get affected by harmful pothole detection, which is not applicable speed for a vehicle
electromagnetic waves, so in this case, the LiDAR system can to run on the road with this speed. Error rate is higher in results
be employed for detection. LiDAR system is a laser scanning due to which the laser-based system is not liable. If the
technique and requires less cost and maintenance. A detection system run with a speed more than 2KMPH, the
computing system is employed based on Raspberry Pi module error rate increases in pothole detection. So, the system
for detection of manholes. Image-based detection technique is becomes irresponsive on the speeds more than 2KMPH and
developed which can perform a number of tasks like becomes unstable. A number of techniques have been
controlling the images and noise filtration while detection proposed based on vision systems to detect pothole by
manholes [11]. employing 2D images and videos [16]. Other researchers
proposed a process of asphalt surface imaging technique [17].
Another innovative smart technology for detecting This method proposed the use of segmentation of region based
manholes is image processing by using a camera and it can
on histogram and spot filters. This process established an
directly be inserted into cars and other vehicles. A number of advanced process to convert images into videos.
image processing modes can be employed based on python
Language from Open CV library and can be installed on cars Therefore, the main objectives of this research is to
and vehicles to detect manholes ahead on the road. One mode develop a manhole detection and monitoring system based on
of image processing technique is the use of smartphone computer vision and image processing techniques in
accelerometer. An automatic report or data can be generated MATLAB environment and to integrate the proposed system
by detecting a manhole in this technique by employing a Web with IOT interface.
server. The proposed system can find out the specific location
of a manhole on the road and provides its information to the II. PROPOSED SYSTEM METHODOLOGY
server, where it is processed and then sends the report to the This research uses a camera (Logitech C170) to captures
user or vehicle driver. An LED light signal is indicated in this frames from road surface. The camera is directly connected to
system on the location of manhole to given an alert to the PC running MATLAB R2020a software. The frames are
driver on road [12]. further processed (by applying Naïve Bayes machine learning
Most of the reviewed studies have proposed their system algorithm) to detect and recognize missing manhole.
in detection and monitoring existence of pothole on the road In order to retrieve the GPR3S location of missing
[11] and [13] while other studies have proposed a monitoring manhole, SIM808 GSM modem is used. The GSM modem is
system based on the pre-known location manhole on the road powered by a 12v battery. In order to stabilize the voltage, a
[7-9]. Therefore, there is gap in this field of research in finding step down converter is used which can regulate the voltage at
or detecting the manholes in the first place. The recognition of 9v DC for the SIM808. The external battery is used in this
manhole on the road requires sophisticated system which can system due to the high power required by SIM808 in
be attached to cars and bikes on the road to report communicating with IOT platform. Arduino Mega is
automatically a missing manhole on road. Detection of connected to SIM808 GSM module. In order for SIM808 to
manhole and differentiation of manhole from potholes is the communicate with Arduino Mega, both boards must have
key challenge to overcome in this study. Finding manhole on same ground. The Rx and Tx pins of SIM808 GSM modem is
road can also drive other features of the proposed system connected to serial 1 (pins 18 and 19, respectively) of Arduino
where the manhole’s lid is missing or the manhole’s lid is not Mega.
properly placed which can prevent many accidents and vehicle
damage that is the major concern in smart cities. The communication between Arduino Mega and PC
(Computer vision block) is using a serial cable. This
Most of proposed systems requires a pre-phase step where communication is designed to be unidirectional from
government shall equip all the manholes with certain systems MATLAB to Arduino Mega only.
proposed in studies of [1]. Therefore, the proposed systems
are expensive in implementation. The major challenge in In the proposed system, there have been TWO (2) main
reduction of this cost is to introduce a portable system that can blocks which include;
scan manholes on the road and report missing or stolen lids. 1) Recognition of manhole through computer-vision
The sensor-based detecting systems are usually developed approach: the camera is utilized in this technique which can
for short range detection, up to 50 cm and with less accuracy capture 30 frames per second and these frames are then
for detection of pothole and manhole. In the study of [14]; and processed to detect the absence of manhole. A Naïve Bayes
[15] a sensor-based system was proposed to detect and report classifier is employed to prepare and filter the frames
the presence of pothole on roads to cars and vehicles. This obtained. MATLAB software is used to flag-up the absence
system is less reliable in terms of safety of the driver and of manhole.
carrier. Sensor-based systems are weak in technology of
2) Localizing the missing manhole: The location of
detection and are not capable to detect the whole surface of
the road. An ultrasonic sensor can detect the distance of a manhole shall be updated on MAP. This feature of system can
carrier from the pothole on road in the range of 2cm to 400cm. be further used by various GPS navigation software app such
The sensor is placed at 30 degrees which can provide 50% as Google Map and WAZE to show the missing manhole on
coverage of the area or a half portion of 400 cm3. This system road to prevent accident. The localization requires system to
is not suitable for a single lane with 3 meters of width. get the coordinates of the missing manhole via the
GSM/GPRS module and update the location on Map on IOT
Laser-based system also contain a number of issues
regarding the speed of vehicle and error in estimation of width platform.
and depth of pothole or opening of a manhole. Study of [11]
 Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 3, (2021) 3

Fig. 1. Circuit diagram of the proposed system

A certain section of the road is cropped through bilateral filter

on a far sight of the camera where asphalt particles are small
to smooth the rest of the road with a same color. It is said to
be a smoothing bilalteral filteration by L*a*b color space. In
case of a small cropped image, processing time can be
enhanced through bilateral filtration process. Better
smoothness can also be attained from a small sized image.
Video Processing (Real-time recognition of manholes): To
detect manholes with a continuous process, frames need to be
processed within less processing time with installed cameras
on cars or roads.

Fig. 2. Manhole cover monitoring and recognition system block

For manhole detection through computer vision system,

the camera need to be placed on the center of a car. The camera
shall be placed at about 30 degrees with respect to the surface
of road on the car.
Naïve Bayes classifier and HOG feature extraction: In the
discussed system, the main machine learning concept is based
on feature extraction of HOG and Naïve Bayes classifier. This
classifier provides more speed in recognition process as
through bicycles the detection of pot-holes could be dismissed
by acceleration. In the given system, camera is set to generate Fig. 3. Block diagram of the computer vision system for Manhole
30 frames per second and total of 17 frames are processed per recognition.
second.
The MATLAB interface provides user to communicate
The undergoing project was supervised through computer- with Arduino by defining the COM port. The upper panel is
vision system and processed through sensor techniques by the wireless network system where user can add the COM port
attaining various samples from road. A manhole can develop and press on “connect To Arduino” button. If the connection
a blob region of 1/8 of the whole frame size. is successful, a green tick sign will appear on the yellow axis
Bilateral filtering to smooth asphalt: The recognition right next to the communication port. The device will
automatically assign the port number for the camera to system
speed could be lower on enhancing the filtration layers and the
asphalt noise can also be reduced through proper filtration. by default.
Therefore, a bilateral filter was employed to smooth the road.
 Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 3, (2021) 4

exists. In some samples, the manhole cover is lid off for the
sake of testing in this project. The duration of this test will be
10 second per samples. The evaluation of the classifier will be
based on following:
 True Negative detection: If the entire test (sample) has
not mistaken in detection while there is manhole cover.
The value of 1 will be appear in the table if this error
happens.
 False Negative detection: Manhole Cover was
missing and it was ignored by program.
 False positive detection: Program flagged missing
manhole cover while the manhole cover was on its
place.
 True Positive detection: Manhole cover was missing
and it was perfectly detected by the program.

ACCURACY OF NAIVE
BAYES…
1
0 2 4 6 8 10
Fig. 4. Manhole recognition system flowchart based on computer vision
Negative Positive

Fig. 6. Naïve Bayes classifier Performance (True = 1 & False = 0)

The result obtained from the accuracy of the Naïve Bayes

classifier indicates that the system can perfectly detect the
missing manhole with total accuracy of 85%. The True
Negative Rate (TNR), Positive Predictive Value (PPV) and
False Discovery Rate (FDR) were found to be 80%, 90%, and
10%, respectively.
B) Misdetection with respect to Car speed
This test is designed to test the accuracy of proposed
system with respect to speed of car. In this test, the car’s speed
increases from 5 to 8, 10, 15, 20, and 30 KMPH. There will
be 5 sample per each speed. In each sample, the accuracy of
detection of manhole cover as well as the accuracy of system
in recognizing missing manhole cover will be observed and
the data will be collected.

Fig. 5. Control panel of system

III. TESTING AND RESULTS

A) Accuracy of pothole detection with computer vision
technique
The test indicates the efficiency and accuracy of the
proposed method in recognition of missing manhole cover.
This test will considers that a car with speed of 20 KMPH is
moving on road to detect missing manhole covers. The speed
of the car was both controlled with the odometer of vehicle
and WAZE application. In this test, a sedan car is used. This
test considers 20 samples. In each sample, manhole covers Fig. 7. Misdetection of system versus the speed of car
 Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 3, (2021) 5

The speed of the car was both controlled with the odometer road, 2) Detect missing manhole, and 3) Update the
of vehicle and WAZE application. In this test, a sedan car is coordinates to IOT platform.
used. Speed of the car increases from 5 to 8, 10, 15, 20, and
30 KMPH. The image processing is activated on MATLAB. The machine learning concept used in this system is based
In this test, the coordinates and IOT update is neglected. In on Naïve Bayes Classifier and HOG feature extraction. The
each observation, the result of system in detecting a manhole main reason to select this classifier is to speed up the process
and missing manhole will be logged on excel file. of recognition as manholes can be dismissed by acceleration
of the car. In this method, total of 17 frames are processed per
The data shows that the system results in 100% accuracy second while the camera is set to resolution of with 30 frames
in detection for speed of below 20 KMPH. However, the per second. This project was based on supervision as the
number of misdetection for manhole cover was raised to 1 for criteria of a possible manhole cover was practically computed
both speed of 25 and 30 KMPH which drops the accuracy of and processed with various samples from road. Usually a
system to 80%. The number of misdetection for missing manhole cover will create a blob region that is 1/8 of the size
manhole cover was also raised to 1 for both speed of 25 and of whole frame. Therefore, small potholes have been ignored
30 KMPH which drops the accuracy of system to 80%. in this system to increase the processing time.
C) Coverage distance of manhole cover detection The result indicates that the camera can varied from 30 to
42 degrees while obtaining best result without any
This test will show what angle is perfect for camera to face misdetection. This can result in detection of manhole in
the road surface. The higher angle will have very poor distance of 5 to 7.6 meters away from car. However,
concentration on road and the manholes in far distance will be increasing the angle to 44 and 46 onwards results in
likely detected (in which it is highly depending on the perfect misdetection mainly due to ignoring the manhole cover.
distance of which the manhole cover is detected in this
system). The best angle must cover 80% of pixels by road REFERENCE
surface. However, if the angle of camera is lowered down a [1] S. H. Ahmed, S. Rani, ‘‘A hybrid approach, Smart Street use case and
lot, the distance of detected manhole will be very close to car future aspects for Internet of Things in smart cities,’’ Future Generation
which may result in misdetection as well. The initial angle of Computer Systems, 79, pp.941-951, 2018.
camera will be 30 degrees and the angle of camera will be [2] T. Eldemerdash, et al., “IoT Based Smart Helmet for Mining Industry
increased up to 46 degrees. This system will take a sample for Application,” International Journal of Advanced Science and
each angle. The initial angle of camera will be 30 degrees and Technology, vol. 29, no. 1, pp. 373-387, 2020.
the angle of camera will be increased up to 46 degrees. The [3] R. Lakshmanan, et al., “Automated smart hydroponics system using
internet of things,” International Journal of Electrical and Computer
system’s performance will be evaluated based on: Engineering (IJECE), vol. 10, no. 6, pp. 6389-6398, 2020.
 Misdetection: System missed a manhole on road or it [4] F. S. Hon, R. Abdulla, S. K. Selvaperumal., “Wheel Chair-Person Fall
wrongly detected a manhole on road while it was not Detection with Internet of Things,” Solid State Technology 63.1s
(2020): 911-922, 2020.
exist.
[5] A. M. Samson, R. Dhakshyani, R. Abdulla., “IOT Based Sustainable
 Total True detected manhole: The missing manhole Wallet Tracking System,” International Journal of Advanced Science
and Technology, 29(1), pp. 1301 – 1310, 2020.
cover was detected correctly.
[6] ALY, H.H., SOLIMAN, A.H. AND MOUNIRI, M., (2015). Towards
a fully automated monitoring system for Manhole Cover: Smart cities
TABLE I. MANHOLE COVER DETECTION COVERAGE DISTANCE and IOT applications. In 2015 IEEE First International Smart Cities
Conference (ISC2), 3(2). pp. 1-7.
[7] G. Jia, G. Han, H. Rao, and L. Shu, ‘‘Edge computing-based intelligent
manhole cover management system for smart cities,’’IEEE Internet
Things J., vol. 5, no. 3, pp. 1648–1656, Jun. 2018.
[8] S. Salehin et al., ‘‘An IoT Based Proposed System for Monitoring
Manhole in Context of Bangladesh,’’ 2018 4th International
Conference on Electrical Engineering and Information &
Communication Technology (iCEEiCT), Dhaka, Bangladesh, 2018, pp.
411-415, doi: 10.1109/CEEICT.2018.8628091.
[9] X. Guo, B. Liu, L. Wang , ‘‘Design and Implementation of Intelligent
Manhole Cover Monitoring System Based on NB-IoT,’’ In 2019
The result indicates that the camera can varied from 30 to International Conference on Robots & Intelligent System (ICRIS),
4(3), pp. 207-210, 2019.
42 degrees while obtaining best result without any
[10] C. Chellaswamy, H. Famitha, T. Anusuya, and S. B. Amirthavarshini,
misdetection. This can result in detection of manhole in ‘‘IoT based humps and pothole detection on roads and information
distance of 5 to 7.6 meters away from car. However, sharing,’’ in Proc. Int. Conf. Comput. Power, Energy, Inf. Commun.
increasing the angle to 44 and 46 onwards results in (ICCPEIC), Chennai, India, 2018.
misdetection mainly due to ignoring the manhole cover. This [11] B.-H. Kang and S.-I. Choi, ‘‘Pothole detection system using 2D
is due to the very poor concentration on road as only 10% of LiDAR and camera,’’ in Proc. 9th Int. Conf. Ubiquitous Future Netw.
pixel is associated to the road. (ICUFN), Jul. 2017, pp. 744–746.
[12] M. M. Garcillanosa, J. M. L. Pacheco, R. E. Reyes, and J. J. P. San
IV. CONCLUSION Juan, \Smart Detection and Reporting of Potholes via Image-
Processing using Raspberry-Pi Microcontroller," in 2018 10th
A remote sensing method is proposed to detect manhole International Conference on Knowledge and Smart Technology (KST),
and recognize missing manhole on road. This method uses Chiang Mai, 2018, pp. 191- 195, 2018.
Naïve Bayes classifier to train dataset and detect as well as [13] K. K. Vupparaboina, R. R. Tamboli, P. M. Shenu, and S. Jana, “Laser-
recognize the missing manhole on road. This research has based de- tection and depth estimation of dry and water-filled potholes:
three main components including 1) Recognize manhole on A geometric approach”, in Proc. Nat. Conf. Communications, pp. 1–6,
2015.
View publication stats

Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 3, (2021) 6

[14] C. Chellaswamy, H. Famitha, T. Anusuya, and S. B. Amirthavarshini,

‘‘IoT based humps and pothole detection on roads and information
sharing,’’ in Proc. Int. Conf. Comput. Power, Energy, Inf. Commun.
(ICCPEIC), Chennai, India, 2018.
[15] S. K. Sharma and R. C. Sharma. ‘‘Pothole detection and warning
system for indian roads,’’ In Advances in Interdisciplinary
Engineering, pages 511–519. Springer, 2019.
[16] A. Akagic, E. Buza, and S. Omanovic, ‘‘Pothole detection: An efficient
vision based method using RGB color space image segmentation,’’ in
Proc. 40th Int. Conv. Inf. Commun. Technol., Electron. Microelectron.
(MIPRO), May 2017, pp. 1104–1109.
[17] K. Azhar, F. Murtaza, M. H. Yousaf, and H. A. Habib, ‘‘Computer
vision based detection and localization of potholes in asphalt pavement
images,’’ in Proc. IEEE Can. Conf. Elect. Comput. Eng. (CCECE),
Vancouver, BC, Canada, May 2016, pp. 1–5.