Journal of University of Shanghai for Science and Technology ISSN: 1007-6735

Remote Control Rover with Camera and

Ultrasonic Sensor
Prof. Rohan Kokate MadhulikaWannewar VarshaPandhare KshitijChoubey
CSE Department IT Department IT Department IT Department
JDCOEM, Nagpur, India JDCOEM, Nagpur, India JDCOEM, Nagpur, India JDCOEM, Nagpur, India
rbk7557@gmail.com mwannewar@gmail.com chandupandhare12@gmail.com choubeykshitijanns@gmail.com

VipulGajbhiye VaibhavPandey
IT Department IT Department
JDCOEM, Nagpur, India JDCOEM, Nagpur, India
vipulgajbhiye03@gmail.com vaibhavpandey0015@gmail.com

Abstract—Disasters and danger come without giving us threats in the surroundings can be countered more efficiently
warning so we must always be prepared for every situation. So [7]. By studying the different development techniques it gives
for handling such situations robots can play an important role the wide area of experimentation [10].
as in these situations there can be a huge risk of life too. It is
wirelessly and remotely controlled through devices in order to
make robots move from source to destination point. In many
cases, the ground becomes uneven due to disaster effects, so
robots can move through such places easily.With a rover it
becomes convenient to search the area without making any
trouble. As in many rescuing operations humans are widely
used to overcome such situations and so for assisting them,
robots can be used for gathering data from accidental places.

Keywords— Advance ip scanner, Putty, VNC, tkinter, OpenCV,

GUI, GPIO.

I.INTRODUCTION
Including natural disasters human-made disaster are also very
dangerous in that there are chances of increasing injured
people so that for identifying their location robot is used
Fig.1. Outer structure of Proposed Rover
[2].As it will go inside the accidental place before other
rescuing team and hence rescuing team will exactly get to
know where they have to find and rescue and due to this they II. LITERATURE SURVEY
do not need to search people in the entire place on their own In [12], "Cost-Effective Autonomous Garbage Collecting
[4]. In such places, the environment is very complex but Robot System Using IOT and Sensor Fusion", authors
robots can easily move and gather information in disaster discussed the system through which they are collecting the
prone areas. Robot has a flexible design as shown in fig no.1 surrounding garbage. As we are also working on a rover so we
which gives it more advantage to move over uneven surfaces. also need to connect different devices and build
It is capable of moving on stairs like structure, road resistance, communication between them, this paper helps a lot. Here the
slope of the ground, inertia resistance. If any obstacle blocks author explained how they developed a 4 wheel rover with a
the path of the robot and if the obstacle is not that much sturdy aluminum sheet and tray for collecting garbage. In this
heighted then it can directly go through the obstacle rather they have used raspberry pi and Arduino uno are used for
than finding a new path. If this is not the case then the robot controlling devices which are attached to rovers such as
can be stopped through the GUI interface given to the user and ultrasonic sensor, motor driver and camera. In this they have
can change the direction from wherever it gets the direction of also discussed how they have improved object detection in
alternate path.With the help of a rescue system, detection of terms of time, memory and speed using a central server for

Volume 23, Issue 7, July - 2021 Page -260

 Journal of University of Shanghai for Science and Technology ISSN: 1007-6735

processing the data. Instead of using on-board processors they such as network structure design and focal loss function
have used servers for processing as on-board processors are construction. They have also mentioned the detailed
having limited processing speed and putting extra load on on- information upon the analysis of both the algorithm and
board processors can slow the other processes too. In this they explained with the help of graphs. HE-SSD model training is
have used a communication model based on Client-Server and also mentioned and details of the environment is also given in
for communicating they have used python's socket which it is carried out. In this comparative analysis is also
programming API. According to the authors their system mentioned through which it is clear that HE-SSD algorithm is
architecture can work easily with more than one rover at a more accurate than SSD algorithm and the results are shown
time as all the rovers will be connected to the central server on in terms of graphs.
the same network. The rover can move autonomously on its
own wheel according to the programmed feature [5]. In [14] ”Live Video Streaming using Raspberry Pi in IOT
Devices”, authors discussed the design and implementation of
In [13] "WI-FI AND ANDROID APPLICATION the technology called Live video streaming using Raspberry Pi
CONTROLLED MULTI-PURPOSE LAND ROVER", in IOT devices, with a single board computer which computes
authors discussed the controlling of rover using Wi-Fi and the Motion Detection Algorithm written in python as
android application with the help of arduino nano and NODE programming environment. They have also explained that this
MCU. In this the authors have developed the robot which is system uses algorithms to significantly decrease the storage
only compatible with android so it will not work on other space and to save the cost and that algorithm is implemented
platforms such as WINDOWS, LINUX and IOS. They have on the Raspberry Pi, which provides live streaming with
solely focused on android technology for user interface from motion detection. They can view the live streaming from any
where every movement and sensors are controlled and web browser or even from mobile in real time. It has a
monitored respectively. In this they have mentioned how they processing system, USB camera, LCD monitor and builds
are collecting data on environmental nature using a variety of necessary peripherals for communication to complete the
sensors such as temperature sensor, humidity sensor, light hardware platform. They applied the Motion detection for a
sensor, Gas sensor and ultrasonic sensor for detecting live streaming camera to analyze the incoming image and
obstacles in the path. For monitoring the surroundings they recognize the movement. The video can save and store the
have used cameras which can rotate up to 0-180 degrees angle image for the review required for the administrators. They
in both the axis for taking view from different directions. So provide the internet access, through cellular connection and
according to the author's all these features are fully virtually Wi-Fi and enable it for the new application. Author has used a
controlled using android application and for joystick control motion detection algorithm that is an automated approach in
for the rover the movements are also done using virtually as which it does not require the individual to start and stop the
physically the joystick is not used. In this the connection is recording. They have decreased the storage and the cost of the
established with the help of the ip address of the robot and the equipment.
android phone and robot are connected on the same network.
In [15] “Improving Efficiency of Ultrasonic Distance Sensors
In [8] "Research on an Efficient Single-Stage Multi Object using Pulse Interval Modulation”, authors have proposed a
Detection Algorithm", authors discussed how they have new operating algorithm for ultrasonic sensors and they are
proposed a high efficient single shot multibit detector (HE- monitoring the algorithm in real time by comparing it with the
SSD) algorithm which is the improved version of SSD object conventional algorithm. Their proposed algorithm has
detection algorithm. They have improved the detection improved the sampling rate of ultrasonic sensors through
process in terms of accuracy of SSD algorithms by designing which signal to noise ratio (SNR) and detection range has also
efficient and dense networks and then after that the robustness improved as compared to conventional operating algorithms.
of the algorithm is improved. In this they have also mentioned As compared to other sensors, ultrasonic distance sensors have
that for detecting small objects more accurately they have much lower sampling rate, that’s why they are hard to apply
used data augmentation. In this they have also explained about on high speed applications. Due to low sampling rate,
SSD algorithm network structure, Loss function and they have ultrasonic sensors are used for finite range only and this
also mentioned where SSD algorithms are lacking and the problem has been resolved by the authors in this paper. In this
areas where improvement or advancement can be made to paper, the author has proposed a novel operating algorithm
increase accuracy. Authors have also discussed the areas and it is analyzed and tested experimentally. For improving
where the advancements are done to improve the accuracy the sample rate, authors have eliminated unnecessary waiting

Volume 23, Issue 7, July - 2021 Page -261

 Journal of University of Shanghai for Science and Technology ISSN: 1007-6735

time in the system which has increased the SNR in the sensor.
In this paper, according to their hardware structure, they have RPi.GPIO library:-With the help of RPi.GPIO library we can
added an analog signal enhancer for and microcontroller is control the GPIO pins of raspberry pi and by changing the
used for enhancing the signal and processing respectively. inputs of different pins the movement of the rover can vary
from one to another. With the help of this library one can set
III. PROPOSED METHODOLOGY
the pins of raspberry pi and use it to perform various
Figure 2.Contains the structure of the system proposed in this operations using functionalities present in the library. We have
paper.
used pins to give I/O. Those pins are GPIO 4, GPIO 17, GPIO
27, and GPIO 22. GPIO 4 is present on pin 7, GPIO 17 is
present on pin 11, GPIO 27 is present on pin 15 and GPIO 22
is present on pin 15.
And the same as 5v is present on pin no.2, GND is present on
pin number 6.

Arduino and Raspberry pi communication: Four sensors

are connected to Arduino Uno and each sensor takes 4 pins,
one for triggering, one for echo, and 2 pins for vcc and ground
respectively and then code has been uploaded in the Arduino
to get it working. In the coding part “Newping” library is used
through which one can measure the distance.
Fig.2. System Structure of proposed rover
First, the code is uploaded in the Arduino to measure the
distance, and then both arduino uno and raspberry pi 3b+ are
Advance ip scanner: It is a free and reliable scanner. It can
connected using USB 2.0 cable type A/B of the Arduino, so
locate all the computers on the local network within a second.
that successful communication could be established between
Firstly an advanced ip scanner tool is used in the project for
them as well as continuous power supply can be provided to
scanning and detecting the ip address of devices present in the
Arduino for working.
current network and in this project we are detecting raspberry
pi ip so that the ip address can be used for controlling the
The "Serial" library is used to print the Output in Raspberry Pi
device remotely.
from Arduino and for that firstly the port at which the arduino
is showing output is set using the "Serial ()" function so that
Putty: It is free and open-source software which gives users
Raspberry Pi can know from where to extract output. After
the option to transfer data securely and also used to upload
seating the port the " readline() " function is used to display
and download files securely in computers. After getting the ip
the outputs of the arduino on the Raspberry Pi output screen.
of raspberry pi using an advanced ip scanner then the ip is
used to check whether the login can be done successfully or
Camera Module: 5MP camera module is attached to CSI port
not. Once the login is successfully done it gets clear that the
of raspberry pi. For capturing real time videos and for loading
connection can be done successfully.
the image in the tkinter GUI OpenCV plays an important role
in it [6]. PIL library is also used to perform various operations
VNC (Virtual Network Computing): It is a graphical
on the frames which are being captured in real time. These
desktop sharing system that enables the desktop display of one
libraries are used to manage the frames and for converting
computer to be remotely viewed and controlled over network
them to the video format.
connection. Once it is cleared that the connection can be made
successfully then VNC viewer is used for monitoring and
Casualty Detection: In this module mainly human detection
controlling the raspberry pi. With the help of VNC viewer we
is done in casualty prone areas and it is achieved using
have controlled the raspberry pi remotely by using python
“Imutils” library through which the captured frame is resized
programming to control the movement of the rover with the
according to the requirement3 HOG or “Histogram of
help of tkinter GUI. We have provided buttons on the GUI
Oriented Gradients” for extracting the data from the
which are used to make the robot move from its initial place.
framesfor detection. “setSVMDetector” functionality to train
On whichever button we click, the robot will move in that
the HOG for detecting human presence in the frames
direction. So this type of mouse functionality is used.

Volume 23, Issue 7, July - 2021 Page -262

 Journal of University of Shanghai for Science and Technology ISSN: 1007-6735

IV. COMPARATIVE ANALYSIS third one is “Start Detect” to detect human presence for
In the table 1, comparison is done between proposed method casualty in the area. Just next to detection button is “STOP”
and Robotic Surveillance Rover. which stops the rover movement and another is “EXIT” which
closes the complete GUI.
Table 1. Comparative Analysis
ROBOTIC PROPOSED ANALYSIS VI. CONCLUSION
SURVEILLANCE ROVER This paper focuses on controlling the robot through tkinter
ROVER[1] GUI interface for the movement of Robot and structure is used
Structure is very Structure is Proposed Rover can
which provides advantage for moving freely on uneven
small in size comparatively big. work more surfaces and for handling the firefighting situations robots can
efficiently on rough be a good option for reducing life casualty. As we are always
ground. putting human life at risk so for handling and analyzing the
situation robots can be a good option for assisting.
One ultrasonic Four ultrasonic Proposed Rover can
sensor is used to sensors are used to detect obstacles REFERENCES
detect obstacles. detect obstacles. from 3 different [1] Kumar Kunal,MohammadSuheab, Keval S Borkar, Kiran HS, “Robotic
directions. Surveillance Rover”,IJARIIE-2018.
[2] MadhulikaWannewar, VarshaPandhare, KshitijChoubey,
It can't climb stairs. It can climb stairs. Proposed Rover VipulGajbhiye, VaibhavPandey, “AI Based Rescue Robot : A Literature
Survey”,JUSST ,2021.
overcomes obstacle
[3] Kim, J.-H., Jo, S., N Lattimer, B. Y. (2016)." Feature Selection for
problems. Intelligent Fire Fighting Robot Classification of Fire, Smoke, and
Thermal Reflections Using Thermal Infrared Images'' Journal of
It can't handle It can handle Proposed Rover is Sensors, 2016, 1–13.
multiple operations. multiple operations. multitasking. [4] Joshua Reich and Elizabeth Sklar, “Robot-Sensor Networks for Search
and Rescue”, Research Gate Paper 2006.
[5] Purnima G, Asst. Prof. AravindS,Renju Mary Varghese, Neethu Anna
V. RESULT Mathew, Gayathri C S, "Alive Human Body Detection and Tracking
System Using an Autonomous Pc Controlled Rescue Robot”, IJETAE
2015.
[6] Chandan G, Ayush Jain, Harsh Jain, Mohana, “Real Time Object
Detection and Tracking Using Deep Learning and OpenCV”, 2018.
[7] Matsuno.F.,andTadokoro,S.(n.d.). “Rescue Robots Systems'' in Japan.
IEEE International Conference on Robotics and Biomimetics.2008.
[8] XinChen,Jing Li , “Research on an Efficient Single-Stage Multi Object
Detection Algorithm”,2019.
[9] Shigeru Kobayashi 1, Yasuhiro Kobayashi 2, Yoshihiro Yamamoto 3,
Taro Watasue 4, Yoshikazu Ohtsubo 5, Takumi Inoue 1, Masanobu
Yasuda 2, ToshiTakamori 6“ Development of a Door Opening System
on Rescue Robot for Search "UMRS-2007".
[10] Michael Sighenza, Diego Guillen,Dante Arroyo, “Mobile Robots
Fig.3. Tkinter GUI controller Development: A Case Study from Robotics Competitions and Course
Projects”.2017.
The tkinter GUI interface and all the functionality in it are [11] GUODONG ZHAI , WENTAO ZHANG , WENYUAN HU , AND
ZHENDONG JI, “Coal Mine Rescue Robots Based on Binocular
working accordingly as they are programmed to react. As in Vision: A Review of the State of the Art”,2020.
the fig.3 you can see that in the upper portion of the interface [12] A.Sengupta, V. Varma, M. SaiKiran, A. JohariandMarimuthu R, “Cost-
real time video is being shown and below that 4 buttons are Effective Autonomous Garbage Collecting Robot System Using Iot And
Sensor Fusion”,IJITEE-2019.
provided which are controlling the forward, backward, left and [13] Maheshbabu P,H Chandra Sekhar& L Ramamurthy, “WI-FI AND
right movement of the rover. Just below the buttons there are 4 ANDROID APPLICATION CONTROLLED MULTI-PURPOSE
LAND ROVER”,IJESRT-2018.
readings showing the distance in cm gathered from different
[14] Dr.G.GSivasankari&Prerana G Joshi, "Live Video Streaming using
ultrasonic sensors from which two sensors are mounted in Raspberry Pi in IOT Devices”,IJERT-2017.
front portion to detect object distance and one sensor is [15] Seungin Shin, Min-Hyun Kim, and Seibum B. Choi, “Improving
mounted at left and right side of the rover. For triggering the Efficiency of Ultrasonic Distance Sensors using Pulse Interval
Modulation”,IEEE-2016.
ultrasonic sensors distance button is given for showing the
readings from the respective sensor. Just above distance
button in the GUI interface there are 5 buttons from which the

Volume 23, Issue 7, July - 2021 Page -263