2019 Conference on Information Communications Technology and Society (ICTAS)

A Smart Home Appliance Control System for Physically

Disabled People
Progress Mtshali Freedom Khubisa
Department of Information Technology Department of Information Technology.
ICT and Society Research Group Durban University of Technology.
Durban University of Technology, Durban, South Africa
Durban, South Africa Email: freedomk@dut.ac.za
Email: progressm@dut.ac.za
Abstract—The construction and conversion of ordinary homes are in bed or siting on a couch. The technology that is used
into “smart homes” has seen a tremendous rise in recent years. by these systems falls under the category of Internet of
This can be ascribed to technologies such as the Internet of Things (IoT).
Things, sensors, smart phones, smart appliances, cloud
computing, and digital assistants such as Amazon Alexa, A smart home is able to notify the residents when a washing
Google Home, Google Assistant, Apple Siri, and Microsoft machine has finished washing clothes, when a roast is cooked
Cortana. At the outset, smart homes were built to and ready to each, show the residents the contents of their
enhance the quality of life for ordinary nondisabled persons. refrigerator remotely, and perform many more tasks that
Impressively, we have seen smart home residents reaping the residents have to manually perform. The cost of a smart home
benefits of security, energy saving, and the ability to control
their lighting, HVAC (heating, ventilation, and air
is generally high but as always; costs go down over time due
conditioning), door locks, and coffee makers while they are in to a number of reasons such as economies of scale or
their space of comfort, for example in bed or siting on a couch. competition among vendors.
However, most smart home devices are not designed with The programmable devices are controlled using smart
people with disabilities and limited range of movement in phones because of their capability to communicate wirelessly.
mind. Of course, being able to control home devices using
Users download and install applications that are designed to
smart technology could be a tremendous benefit to people with
physical disabilities and the older persons. This paper presents send commands to the programmable devices wirelessly.
a system that uses smart plugs, smart cameras, smart power Smart home residents are also able to send commands via
strips and a digital assistant such as Amazon Alexa, Google computers. Basanta [3] proses a system that uses voice and
Home, Google Assistant, Apple Siri, or Microsoft Cortana to gestures to control the programmable devices in a smart
capture voice commands, from a person with physical
home. Augmented reality has also been used to allow users to
disabilities, spoken in a much more natural way to control
ordinary home electrical appliances in order to turn them on control smart devices [4].
or off, with minimal exertion. Being able to control home devices using smart technology
could be a tremendous benefit to people with physical
Keywords – smart homes; digital assistants; voice disabilities and the older persons [5]-[7]. Most of the smart
activated system; smart plugs. home devices are not designed with people with disabilities
and limited range of movement in mind. We can agree that
I. INTRODUCTION “disabled persons face many problems in their daily life
both inside and outside of the home especially while staying
The so-called “smart homes”, also referred to as
alone at home” [8].
“connected homes” are fast becoming an interesting and
In this paper, we present a system that uses a digital
growing global phenomenon. Waleed [1] defines a smart
assistant such as Amazon Alexa, Google Home, Google
home as “living environments supplied with advanced
Assistant, Apple Siri, or Microsoft Cortana to capture voice
intelligent technologies that manipulate and respond
commands from a person with disabilities spoken in a much
according to the requirements of the home residents”.
more natural way to control ordinary electrical appliances to
Automation forms a large part of smart homes. Connected
turn them on or off. The digital assistant, in the case, Amazon
programmable devices utilize “a network to control home
Alexa, can communicate with a smart electrical power strip
functions for a better quality of life” [2] and more
or smart mini power socket via Wi-Fi to turn the electrical
convenience. These devices use wireless technology such
appliances plugged into them on or off. The person with a
as Wi-Fi (IEEE 802.11), Bluetooth (IEEE 802.15.1), Radio
disability simply asks Alexa to turn an appliance on or off by
Frequency Identification (RFID), ZigBee, IPv6 Low-Power
simply saying “Alexa turn on the microwave”, for example.
Wireless Personal Area Network (6LoWPAN), and more.
The limitations of the smart home programmable devices
Smart home residents are able to control lighting, control
can be overcome by designing systems that make use of the
their HVAC (heating, ventilation, and air conditioning),
ability to send commands to these systems using voice if the
lock and unlock doors, turn on the coffee make while they
ISBN 978-1-5386-7365-2/19/$31.00 ©2019 IEEE
disabled person can speak or use gestures to communicate RF receiver is connected to the Raspberry Pi (RPi). The RPi
with the system. Additionally, smart phones could be used by is connected to the temperature, humidity, motion detectors
a person with disabilities to control the appliances plugged and other sensors. The data captured from the sensors is sent
into a smart plug. The paper focuses on using digital assistants to the cloud for processing and access by users. The
to turn the appliances on and off. The rest of the paper is disadvantage of eHome is that users have to learn commands
organized as follows. Section II discusses the current state of in order to control the system and the complexity of the
the art in the field of smart homes, particularly of disabled system makes it harder to use especially by people with
people. Section III presents the design of our system disabilities.
architecture and describe scenarios that are ideal for our Sunehra and Tejaswi [6] propose a system that consists of
solution. Section IV provides the conclusion of the designed two schemes. The first scheme uses the HC-05 Bluetooth
architecture and highlight some future works. module and an Arduino Bluetooth controller mobile
application to control appliances (i.e. turn them on an off).
II. LITERATURE REVIEW AND PREVIOUS WORK The second scheme uses Global Systems for Mobile
Communication (GSM)/General Packet Radio Service
The system designed by Iqbal at al. [4], uses Microsoft
(GPRS) to control electrical appliances. The sensors,
Kinect and the X10 protocol to design a system for the people
Bluetooth module, and SIM 900 GSM modem are connected
with disabilities so that they can control smart home
to the Raspberry Pi as the microcontroller. Relays that control
appliances using gestures. Their system’s architecture
lights, and other modules receive commands from the RPi.
consists of a Kinect connected to the Central Control Unit
Web Cams can also be connected to the RPi. Voice
(CCU) (which is a desktop computer). The CCU is connected
commands are sent to the RPi using an Arduino BlueControl
to an X10 transceiver module through a USB/Radio
application that is installed on a smart phone. The end user
Frequency. The X10 module sends commands through
uses the smart phone to send commands to the RPi. We
existing wires to the appliances or devices. There is also a
contend that this system is complex for a person with
database that stores the addresses and state of each of these
disabilities and could also be expensive compared to our
appliances. The disabled person simply points to the device
solution.
they want to control. The Kinect sensors pick up the gestures
Basanta et al. [3] propose a system that uses both voice and
and sends them to the CCU to be converted to commands for
gestures. It uses the MPU6050 accelerometer to capture
the appliance. The disadvantage of this system is that it only
gestures instead of the Microsoft Kinect as in the system in
works for people who are unable to speak.
[4]. Voice is captured using dedicated hardware components
Rashid et al. [8] propose a system for the home automation
such as Bluetooth, Arduino boards, smartphone, and an AMR
for the physically disabled people. Its architecture includes an
(adaptive multirate) audio codec [3]. The home appliances are
array of IR (infrared radiation) sensor modules, an ultrasonic
connected to a microcontroller along with Bluetooth devices.
sensor (HC-SR04) module, Bluetooth module, a
Users are able to turn appliances on and off using smart
Microprocessor Unit (MU) (an Arduino Mega2560), and
phone. The gesture component of the system consists of the
an array of relays that are connected to appliances. The
MPU-6050 accelerometer, a Nano microcontroller
sensors feed the state data of the appliances to the MU which
ATMEGA328P, and Bluetooth HC-06 serial module to
uses the relays to trigger the appliances. Doors automatically
collect accelerometer data when the use waves their arms and
open and close when people approach the door. The primary
hands. The gesture scheme requires that filters be used to
objective of this system is to reduce water usage (pumps
filter noise, verify and smooth data using mathematical
controlled via the MU) and power consumption. The
manipulations. Again, our solution is simpler and cheaper
disadvantage of this system is that it is not specifically
because no noise filtration is necessary and end users do not
designed for people with disabilities since they do have direct
need to use a smart phone to issue commands. The user can
control of the appliances through the system controls.
use normal conversational phrases/commands to control
Vineeth at al. [7] propose a system called eHome. Their
appliances via the digital assistant.
system is somewhat similar to the system in [8] except that
The system that comes closer to what we are proposing is
theirs uses voice commands to control appliances. People
designed and developed by Yue and Ping [2] shown in Figure
with physical disabilities can use the system. The eHome
1. A voice or text command is sent to the Amazon Alexa
architecture includes a voice recognition module V3, a radio
frequency (RF) modules that consists of a transmitter and
receiver, an Arduino UNO microcontroller, a Raspberry Pi
Model B+, a microphone, and power supply for Arduino and
Raspberry Pi. The user uses the microphone which is
connected to the V3 module to issue commands. The V3
module is connected to the Arduino UNO. It converts
commands into internal data structures that are sent to the
Arduino UNO which transmits them to the RF receiver. The
 Figure 2. Alexa-enabled appliances.

Our system takes advantage of smart plugs, smart cameras,

smart video cameras and smart power strips that are Wi-Fi
enabled and Alexa-enabled. The idea of using smart plugs has
been used before to monitor power and managing problems
[10]-[13]. The smart video cameras such as DeepLens can be
used as an additional authentication feature to capture human
presence inside the premises or perform face recognition and
thus mitigating remote attacks such as voice squatting attack
and voice masquerading identified in [17]. The Alexa device
Figure 1. Yue and Yang [2] Voice activated system is activated to accept voice commands once the DeepLens
detects a legitimate person. A similar idea is propounded in
Service. The service interprets commands and sends [14] where they use a virtual security button based on Wi-Fi
commands to specific GPIO (General-Purpose Input/Output) technology to detect the presence of a human with the goal of
pins on the Raspberry Pi. The home or household appliances enabling Alexa to accept commands only from authentic
then receive commands from the RPi wirelessly. They could persons. The Amazon Echo is also Alexa-enabled and can
be connected to the RPi via a wired network. thus interact with the smart plug. The Users can ‘talk’ to
We have discussed several solutions that have been Alexa via the Amazon Echo and instructs ‘her’ to send
proposed to enhance the lives of people with disabilities by commands to the smart plug or power strip to turn the device
making it easier for them to control modern devices that are or devices plugged into them on or off.
not necessarily smart devices using their voices, gestures, and
smart phones. Our system is simpler, user-friendly, Consider Figure 3 for our system. Suppose that the person
costeffective, and easy to implement. with a walking disability has a regular (non-smart) stove,
slow cooker, lamp, fan, microwave oven, toaster, refrigerator,
washing machine, and a toaster oven.
III. OUR SYSTEM’S ARCHITECTURE
The introduction of digital assistants such as Amazon
Echo, Amazon Echo Show, Google Home, Google Assistant,
Microsoft Cortana, IFTTT (if this than that), and many others,
has made interacting and controlling appliances much easier
using voice commands [2]. Additionally, devices and
appliance manufactures have designed devices that can
interact with digital assistants thus making it easier to control
or send controls to them via commands. Some of such devices
shown in Figure 2, include Phillips Hue light bulbs,
thermostats like Ecobee, Nest, others, smart plugs,
refrigerators, smart cameras, smart video cameras, door
locks, Wemo light switches, and many more. These are Alexa
enabled and also work with Google Home, Google Assistant.
Alexa-enabled means that the device can receive commands Figure 3. Using Alexa to control regular appliances via the Alexa-enabled
from Amazon Alexa. smart plugs

Let us consider three possible scenarios:

Scenario 1
The regular appliances may be plugged in the Alexa
enabled power strip or the mini smart plugs that can be
plugged into a regular power wall socket. In Figure 3, the
Alexa-enabled power strip has four smart sockets that can be
individually and independently turned on and off. The stove
is plugged in socket 1, the slow cooker in socket 2, the lamp
in socket 3, and the microwave oven. The toaster, refrigerator,
washer, and toaster oven are plugged in the smart wall
sockets. The smart plugs support Wi-Fi and are configured to
communicate with Alexa using smart phone apps that are
downloadable from Google Play and Apple Store (see Figure
2). The sockets can be configured to use names. For example,
socket 1 could be named “stove”, socket 2 be “slow cooker”,
etc. The person could simple say:

“Alexa, turn on the stove”,

“Alexa turn off the slow cooker”
“Alexa turn on the lamp”, etc.
Scenario 2
In Scenario 1, the person with a walking disability can only
control regular appliances using the smart plugs but would
also like to have surveillance of the premises inside and/or
outside. There are Alexa smart cameras such as the Amazon
Cloud Cam Security Camera, Wyze, Kasa, and many more. Figure 5. Using DeepLens video camera monitor, detect and respond to
The camera would be connected to a TV or computer screen. unpleasant incidents.
IV. CONCLUSION

The lives of people with disabilities do not have to be

hopeless just because of their physical limitations.
Technology, for a long time, has not been focused on how it
can be used to better their lives so that they can enjoy a better
quality of life. A number of researchers have
attempted to rectify this problem by applying IoT solutions
for the betterment of their lives. However, such solutions
have been complex, cumbersome, and expensive for older
persons and physically disabled people. When one considers
the cost of these smart devices, one quickly realizes that
people with disabilities are unable to afford them, especially
if they live alone on a fixed budget. The system presented in
Figure 4. Premises surveillance using smart cameras. this paper, is much simpler, easy to use, makes the interaction
between a person with disabilities and the digital assistant
The person with a walking disability would ask Alexa to show more natural, and is much less expensive. The system
different views of the premises via the camera by simply presented in this paper could also be easily adapted for use by
saying: the elderly people who have limited mobility. Since our
system essentially relies on voice commands, which may be
“Alexa, show me outside” “Alexa, show me the bedroom” realized as a shortcoming for persons who cannot speak or
uses sign language for communication, we aim to integrate a
Incidentally, the camera has a two-way audio. If someone was wearable device for the purpose of provisioning for both
at the door, the person with the disability could see them voice commands and gesture movements in our system.
through the camera and talk to the person at the door without
opening the door. Note that the camera is not connected to the REFERENCES
smart plug since it works directly with Alexa.
[1] J. Waleed, A. M. abduldaim, T. M. Hasan, and Q. S. Mohaisin, “Smart
Scenario 3 Home as a New Trend, a Simplicity Led to Revolution” in 2018 1st
International Scientific Conference of Engineering Sciences – 3rd
It is accentuated that falls are the main source of accidental Scientific Conference of Engineering Sciences (ISCES), pp. 30-33.
death in elderly persons [15], especially for those who are [2] C. Z. Yue, and S. Ping. “Voice Activated Smart Home Design and
living alone. In Figure 5, the DeepLens can be configured to Implementation” in 2017 2nd International Conference on Frontiers of
Sensors Technology, pp. 489-492.
detect when a physically disabled or elderly person falls and [3] H. Basanta, Y. Huang, and T. Lee, “Assistive Design for Elderly Living
respond in real-time by sending a voice command, using Ambient using Voice and Gesture Recognition System” in 2017 IEEE
Audio out feature, to Alexa; instructing her to call a nearby International Conference on Systems, Man, and Cybernetics (SMC),
care center or institution. Alexa can use “her” location as an Banff Center, Banff, Canada,
incident location to enable quick assistance. The DeepLens [4] A. Iqbal, M. Arifin, and A. Hossain, “Smart Home Appliance Control
System for Physically Disabled People using Kinect and X10”, in 2016
video camera analyzes what it sees and takes necessary 5th International Conference on Informatics, Electronics and Vision
actions [16]. (ICIEV), pp. 891-896.
[5] D. H. Stafanov, Z. Bien, and W. Bang, “The Smart House for Older
Persons and Persons with Physical Disabilities: Structure, Technology
Arrangements, and Perspectives. IEEE Transaction on Neural Systems
and Rehabilitation Engineering, vol, 12, no. 2, pp. 228-250, June 2004.
[6] D. Sunehra, and V. Tejaswi, “Implementation of Speech Based Home
Automation System using Bluetooth and GSM” in International
Conference on Signal Processing, Communication, Power and
Embedded System (SCOPES)-2016, pp. 807-813.
[7] K. V. Sai, B. Vamshi, and V. K. Mittal, “Wireless Voice-Controlled
Multi-Functional Secure eHome”, IEEE Transactions, pp. 2235-2240.
[8] H. Rashid, S. B. Osman, N. Hassan, I. U. Ahmed, R. Das, and M.
Karim, “A New Design Approach of Home Automation System for
Patients with Physical Disability to Reduce Water Wastage and Power
Consumption using Renewable Energy” in Proceedings of the 2017 4th
International Conference on Advances in Electrical Engineering, 2830
September, 2017, Dhaka, Bangladesh, pp. 770-774,
[9] Y. Thongkhao, and W. Porn, “A Low-cost Wi-Fi Smart Plug with
Onoff and Energy Metering Functions”, IEEE Transactions.
[10] A. S. Musleh, M. Debouza, and M. Farook, “Design and
Implementation of Smart Plug: An Internet of Things (IoT) Approach”
in 2017 International Conference in Electrical and Computing
Technologies and Appliances (CECTA), no page numbers.
[11] T. Petrovic, and H. Morikawa, “Active Sensing Approach to Electrical
Load Classification by Smart Plug”, IEEE Transactions, no page
numbers.
[12] M. Choi, W. Park, and I. Woo, “Smart Office Energy-Saving Service
using Bluetooth Low Energy Beacons and Smart Plugs” in 2015 IEEE
International Conference on Data Sciences and Data Intensive
Systems, pp. 247-251.
[13] M. Chen, and C Lin, “Design and Implementation of a Smart Home
Energy Saving System by Multi-Microprocessors” in
2015International Conference on Consumer Electronics-Tainwan
(ICCE-TW), pp.410-411.
[14] L. Xinyu, G. Tu, A. X. Liu, C. Li, and T. Xie. "The insecurity of home
digital voice assistants-amazon alexa as a case study." arXiv preprint
arXiv:1712.03327 (2017).pp. 1-13
[15] L. Fernández, H. A. Breinbauer, and P. H. Delano, “Vertigo and
dizziness in the elderly”. Frontiers in neurology, vol. 6, p.144.
[16] https://docs.aws.amazon.com/deeplens/latest/dg/what-is-
deeplens.html
[17] N. Zhang, X. Mi, X. Feng, X. Wang, Y. Tian, and F. Qian,
“Understanding and Mitigating the Security Risks of Voice-Controlled
Third-Party Skills on Amazon Alexa and Google Home”. arXiv
preprint arXiv:1805.01525.