COLLEGE OF COMPUTING AND INFORMATICS TECHNOLOGY

COMMUNICATION AID FOR THE DEAF AND MUTE

By

CS22-2
DEPARTMENT OF COMPUTER SCIENCE

SCHOOL OF COMPUTING AND INFORMATICS TECHNOLOGY

A Project Report

submitted to the School of Computing and Informatics Technology

in Partial Fulfilment of the

Requirements for the Award of the Degree of Bachelor of Science in Computer Science

Of Makerere University

Supervisor
MADAM ROSE NAKIBUULE

School of Computing and Informatics Technology, Makerere University

rnakibuule@cit.ac.ug, +256-701116829

OCTOBER, 2022

ii
 DECLARATION

We, Kateregga Alex, Ssemakula Jonathan, Noowe Charlotte and Kabba Joseph Timothy
hereby declare that this report, written in partial fulfilment of the requirements for the award of a
Bachelor of Science in Computer Science degree at Makerere University, is our very own
authentic work and the content of this document has never been submitted to any institution.
Note however, that Citations, Quotations and References to other people's work or sources of
information where used, have been duly made.

Email address Signature

Registration
Names Number Student Number

katereggaaalex07
KATEREGGA 19/U/16571/PS 1900716571 07@gmail.com
ALEX

uhurussemakula0
SSEMAKULA 19/U/0557 1900700557 69@gmail.com
JONATHAN

kabbajoseph072@
KABBA 19/U/16467/EVE 1900716467 gmail.com
JOSEPH
TIMOTHY

noowecharlotte@
NOOWE 19/U/17693/PS 1900717693 gmail.com
CHARLOTTE

iii
 SUPERVISORY APPROVAL
I, Rose Nakibuule, have been working with this group as their direct supervisor on this
“Communication aid for the deaf and mute” final year project and I am confident that the work
presented here is credible and worthy of a computer science final year project.

The team has exhibited excellent research and implementation skills as expected of a competent
computer science student. I therefore give this project my full supervisory approval to proceed to
the next stage of evaluation.

Signature………………………………………. Date………………………………………

iv
 DEDICATION
We dedicate this Project and this report to our Parents, Guardians, Mastercard foundation
Scholar’s program and lecturers that have made it possible for us to reach this level of University
Education by selflessly providing for all our needs, financially, emotionally and in other forms.
May God Bless You Abundantly.

v
 ACKNOWLEDGEMENTS

First and foremost, we would like to acknowledge the Almighty God who has guided us throughout
the three years of the degree programme.

A heartfelt appreciation goes to our parents, Mastercard foundation scholars’ program and
guardians for their consistent support during the pursuit of our Bachelor’s Degree.

We are grateful to our academic supervisor, Madam Nakibuule Rose, for the professional guidance
and direction rendered to us whenever we needed it and for being available to us regardless of her
busy schedule. You made this entire project run more smoothly than we could have ever imagined.

We are also very thankful to the members of Makerere University Disability Students’ Association
(MUDSA) who gracelessly participated in our research studies and provided us with the necessary
information for our project. Thank you for bearing with our demands even when you there were
other things that you rather have spent time doing.

Finally, we are very grateful to each of the team members for the spirit of teamwork and
consistency that was exhibited throughout the course of this project. Thank you for encouraging
and bring out the best in each other

vi
 ABSTRACT
Although many gadgets have been developed for easy communication, communication is still a
challenge for impaired people like the deaf and mute. This is because most of the gadgets and
applications work online and some gadgets are too expensive and yet the majority cannot afford
them.

This project investigates how an offline communication aid, with a high level of usability can be
designed to solve the problem. This system translates English speech to text and text to speech.
The research process consisted of a literature study for gathering all the requirements for such a
system and the development of a prototype, whose usability has been evaluated, as well as the
willingness of people to adopt this innovation using observational methods such as questionnaire
survey method. Results are presented in this report and conclusions are drawn. The study shows
that most of the respondents are willing to adopt this product.

vii
Table of Contents
CHAPTER ONE ............................................................................................................................. 1
INTRODUCTION .......................................................................................................................... 1
1.0 BACKGROUND ................................................................................................................ 1
1.2 PROBLEM STATEMENT ................................................................................................. 4
1.3 OBJECTIVES ................................................................................................................... 4
1.3.1 MAIN OBJECTIVE ..................................................................................................... 4
1.3.2 SPECIFIC OBJECTIVES ........................................................................................... 4
1.4 SCOPE ............................................................................................................................. 4
1.5 SIGNIFICANCE ................................................................................................................ 4
CHAPTER TWO ............................................................................................................................ 5
LITERATURE REVIEW ............................................................................................................... 5
2.1 REVIEWED SYSTEMS .................................................................................................... 5
2.1.1 CAMERA-BASED SYSTEMS..................................................................................... 5
2.1.2 OTHER SYSTEMS .................................................................................................... 6
2.1.3 TRAINED MODELS ................................................................................................... 8
CHAPTER THREE ........................................................................................................................ 9
RESEARCH METHODOLOGY/ RESEARCH DESIGN ............................................................. 9
3.2 OVERVIEW ...................................................................................................................... 9
3.3 RESEARCH METHODOLOGY........................................................................................10
3. 3.1 RESEARCH PROCESS ...........................................................................................10
3.3.2 DATA COLLECTION AND ANALYSIS ......................................................................11
3.4 SYSTEM DEVELOPMENT METHODOLOGY .................................................................13
CHAPTER FOUR ......................................................................................................................... 15
RESULTS/ FINDINGS ................................................................................................................ 15
4.1 DATA COLLECTION ..................................................................................................15
4.2 PROPOSED SYSTEM ................................................................................................21
4.2.1Hardware components ...............................................................................................21
4.2.2 Software Requirements .............................................................................................25
Re training of the deepspeech-tflite model .........................................................................27
4.3 CONCLUSION ............................................................................................................28
CHAPTER FIVE .......................................................................................................................... 29

viii
DISCUSSION OF RESULTS/ FINDINGS .................................................................................. 29
5.1 RESULTS FROM THE SURVEY .....................................................................................29
5.1.1 CHALLENGES FACED DURING RESEARCH..........................................................29
5.2 LIMITATION OF THE SYSTEM .......................................................................................30
CHAPTER SIX ............................................................................................................................. 31
CONCLUSION ............................................................................................................................. 31
CHAPTER SEVEN ...................................................................................................................... 32
RECOMMENDATIONS .............................................................................................................. 32
REFERENCES ............................................................................................................................. 33
APPENDICES .............................................................................................................................. 37
APPENDIX A .........................................................................................................................37
APPENDIX B .........................................................................................................................38

ix
 LIST OF FIGURES

Figure 1 pie chart showing gender Distribution ...........................................................................15

Figure 2 Bar Graph showing age distribution .............................................................................16
Figure 3 Bar Graph showing age distribution by gender ..............................................................16
Figure 4 Bar Graph showing Current Level of education ............................................................17
Figure 5 Bar Graph showing School or institute attended by interviewee .....................................17
Figure 6 Bar Graph showing district distribution ........................................................................18
Figure 7 Pie chart for Care taker ................................................................................................18
Figure 8 Pie chart showing level of hearing .................................................................................19
Figure 9 Pie chart showing means of communication ...................................................................19
Figure 10 Pie chart showing Basis of Payment.............................................................................20
Figure 11 Pie chart showing need for our proposed system ..........................................................20
Figure 12 System block diagram .................................................................................................21
Figure 13 Raspberry pi 4 model B ..............................................................................................22
Figure 14 speaker connect by Bluetooth Figure 15 Speaker connected by cable ...........24
Figure 16 Touch Screen case for Raspberry pi ............................................................................24
Figure 17 showing microphone connected to Raspberry Pi ..........................................................25
Figure 18 Output for text to speech (for the mute) .......................................................................27
Figure 19 output for speech to text (for the deaf) .........................................................................28

x
 LIST OF TABLES

Table 1 Showing Research Objectives and Methodology ............................................................... 9

Table 2 Specifications of the raspberry pi 4 model B ...................................................................23
Table 3 Performance Results ......................................................................................................28

xi
 LIST OF ACRONYMS OR ABBREVIATIONS
AI Artificial intelligence
ARM Advanced RISC Machine
CDC Centres for Disease Control and Prevention
CRPD Constitution of the Republic of Uganda, the United Nations Convention on the
Rights of Persons with Disabilities.
Govt Government
MDAs Ministries Departments and Agencies
NCD National Council for Disability
NDP National Development Plan
SDGs Sustainable Development Goals
SPI Serial Peripheral Interface
UBOS Uganda Bureau of Statistics
UML Unified Modelling Language
UN United Nations
UNAD Uganda National Association of the Deaf
UNASLI Uganda National Association of Sign Language Interpreters
UNDP United Nations Development Programme
WHO World Health Organisation
DBMS Database Management System
MUDSA Makerere University Disability Students’ Association
IDE Integrated Development Environment
OS Operating System

xii
CHAPTER ONE

INTRODUCTION
In this chapter, we describe the problems encountered by the deaf and mute when
communicating with other people while at home, school, workplaces and other environments,
objective of the study and why it is important to solve these problems.

1.0 BACKGROUND
With advances in information technologies, mobile communication devices have become
important tools to connect with people and these have simplified the communication among the
people, bringing them closer to each other. Unfortunately, communication among the disabled is
lagging behind despite the improvements in technology.

According to National Centre on Birth Defects and Developmental Disabilities, Centre for
Disease Control and Prevention (2020), Disability is any condition of the body or mind
(impairment) that makes it more difficult for the person with the condition to do certain activities
(activity limitation) and interact with the world around them (participation restrictions). There
are many types of disabilities, such as those that affect a person’s:
• Vision
• Movement
• Thinking
• Remembering
• Learning
• Communicating
• Hearing
• Mental health
• Social relationships

Although “people with disabilities” sometimes refers to a single population, this is actually a
diverse group of people with a wide range of needs. Two people with the same type of disability
can be affected in very different ways. Some disabilities may be hidden or not easy to see.

According to the World Health Organisation (WHO, 2001), disability has three dimensions:
1. Impairment in a person’s body structure or function, or mental functioning. Examples of
impairments include loss of a limb, loss of vision or memory loss.
2. Activity limitation, such as difficulty seeing, hearing, walking, or problem solving.

1
 3. Participation restrictions in normal daily activities, such as working, engaging in social
and recreational activities, and obtaining health care and preventive services.

Persons with Disabilities are still among the most neglected groups in the policy domain as well
as in the private sphere. The majority of these persons face enormous economic, political, and
social barriers that have an adverse impact on their physical, economic, social and intellectual
development and well-being. Many of them do not have access to basic needs such as health,
education, experience multiple deprivations even within their family and are invisible in the
national policy agenda.

The disability status report-Uganda 2019 by National Council for Disability(NCD, 2019),
highlights the contributions made by the Ministries Departments and Agencies (MDAs) and
other development partners to promote national objectives and articles on inclusion and non-
discrimination as provided in the national and international legal framework: For example, the
provisions enshrined in the 1995 Constitution of the Republic of Uganda, the United Nations
Convention on the Rights of Persons with Disabilities (CRPD)(Centre for Human Rights, 1995;
Uganda, 2016); the National Policy on Disability in Uganda (2006) among others.

In Uganda, the 2014 population and housing census report (UBOS, 2016) indicated that 12.4%
of the Ugandan population lives with some form of disability hence making it a development
concern. This nearly correlated with the World Health Organisation Report of 2010(WHO, 2010)
which put the global disability prevalence rate at 15%. Disability is currently recognized as a
development concern that requires inclusion as noted in the Sustainable development Goals
(SDGs); National Development Plan (NDP II) 2015/16 – 2019/20(Uganda Govt, 2015) and; the
Social Development Sector Plan (SDSP1) 2015/16– 2019/20(NDP, 2015; Ministry of Gender
Labour and Social Development (MoGLSD), 2016) among others.

The categorization of disability in Uganda is enshrined in schedule B of the NCD(amendment)

Act (2013) provides for disability coding to include: sensory disabilities – Deaf with speech,
Deaf without speech, Deaf-blind, Hard of hearing, total blindness and, low vision (nose readers);
physical disabilities – amputations, one-arm, both arms, one leg and, both legs; Deformities –
lower limb, upper limbs, shoulders, forearm and hand, club feet, spina bifida, spinal cord
injuries, rheumatism , little persons and persons living with albinism.

According to World Health Organisation estimates (2005), 278 million people worldwide have
moderate to profound hearing loss in both ears. In Uganda, more than 1,080,000 people are Deaf
(UBOS, 2014). 90 % of deaf people in East and Central Uganda have never been to school. They
live in absolute poverty, powerless, isolated and voiceless. As a result, they are not employed,
have no access to information and limited access to social and economic services (UNAD, 2012).

2
Uganda is currently ranked 159 out of 189 in the 2019 UN Human Development report (UNDP,
2019) and is a signatory to both Education for All and the Convention on the Rights of Persons
with Disabilities. Whilst huge strides were made towards reaching the MDGs in education, the
pace of progress was not sufficient to enable all Ugandan children to enrol and complete a full
primary education. For children with special education needs the situation is worse. It is
estimated that around 65% of deaf children fail to access primary education, and hearing children
are statistically still four times more likely to attend school than deaf children (Sign Health
Uganda, 2018).

There are interventions that have been made to help deaf and dumb communicate for example
there are human interpreter who help to interpret for the dumb and deaf and normal person but
this has not been effective because the number of available interpreters is few compared to
population of the deaf and dumb, another concern is the interpreter’s efficiency regarding
privacy and whether or not information is altered and the cost of training new interpreters is
high. Mobile applications and handheld devices have been developed to aid but these systems are
online and some of these devices are bulky. Camera-based systems to help in text reading frame
to help visually impaired people read text labels, sign language recognition system and speech
recognition technology that help the deaf and dumb communicate with the normal people have
been developed but these systems are online systems that require data connection and in a
country like Uganda where there’s poor network coverage, this becomes a problem. This means
that the deaf and mute in areas without internet coverage and those that cannot afford to pay the
internet service providers are at a disadvantage and will not be able to operate the online system.

Currently, the deaf and mute do not have an offline system in place to use to communicate to
people that do not understand sign language. The common medium used to communicate to
others is through a human interpreter and use of online interpretation systems. This raises
concerns with the fee required by the interpreters and according to UNASLI and Pay Lab (2008),
the average monthly salary for a sign language interpreter is 1.7 million of which very few
families can afford to pay this per month. This has therefore affected the deaf and mute right
from their childhood in many ways including no access to education, unemployment,
marginalization and discrimination in societies, and lack of privacy since they have to be with
middlemen for effective communication.

Therefore, we intend to investigate the current communication aid systems for the deaf and dumb
so as to pinpoint the setbacks that could be causing such a communication gap and thereafter
develop a system that would promise to bridge the communication gap between the deaf, mute
and normal people.

3
1.2 PROBLEM STATEMENT
Despite the various technologies such as mobile gadgets and mobile applications that have been
developed to aid hearing impaired individuals in communicating with each other and with people
without disabilities, communication is still a challenge among the deaf and mute. This is because
most of the systems do not support offline usage and some systems are bulky. In addition to that,
the systems are costly, and most of them are mobile based applications and these systems cannot
work in areas where people do not have smart phones and for this reason very few can afford
thereby leaving a big communication gap.

1.3 OBJECTIVES

1.3.1 MAIN OBJECTIVE

To develop an offline system that would act as a communication aid for the deaf and mute.

1.3.2 SPECIFIC OBJECTIVES

1. To study the current communication aid systems for the deaf and mute and identify areas
that would need improvement
2. To translate speech to text.
3. To translate text to speech.
4. To implement then evaluate the prototypes and the user experience of the system.

1.4 SCOPE
The scope of this project is to develop an offline English text to speech and speech to text
translator.

1.5 SIGNIFICANCE
The system would ensure easy communication among the deaf and dumb in the schools and
community at large which in the long run will improve their livelihood.

This system would help the impaired people to have communication with each other and also
with the normal person.

This system would help eliminate the costs paid to internet service providers since it works
offline.

4
CHAPTER TWO

LITERATURE REVIEW
During the Literature Review, we reviewed articles and systems developed by different people to
address the same problem we intend to solve.

2.1 REVIEWED SYSTEMS

To address the problems of deaf, dumb and blind people, the following authors carried out the
following work;

2.1.1 CAMERA-BASED SYSTEMS

These systems use cameras to capture text and some capture gestures made and translate them
into text which is read by the system.

2.1.1.1 CAMERA-BASED HELP TEXT READING FRAME

Chucai et al. (2013) have developed a camera-based help text reading frame to help visually
impaired people read text labels and handheld devices. This article describes a scheme that reads
text printed on portable objects to aid the visually impaired person. A text-to-text support
framework helps the visually impaired in their daily lives. One can create custom images with
easy-to-use and easy-to-use tools. A region of a moving object is selected as a background
subtraction method using a mixture of Gaussian. Location and text recognition are performed to
obtain textual information in a separate ROI (Range Operating Instruction). They proposed a
new text localization algorithm, an object-oriented image processing algorithm. Text characters
in the restrained text area are binarized and recognized by commercial software OCR (Optical
Character Recognition). Recognized text is then sent to users with visual impairments.

2.1.1.2 VISUAL ASSISTANCE SYSTEMS FOR LABEL DETECTION VIA

VOICE OUTPUT
"Visual Assistance Systems for Label Detection via Voice Output" proposed by Vasanti and
Ramesh (2014), reads additional text based on a camera that allows the visually impaired to read
text labels and product packaging in their daily lives. A webcam connected to a Raspberry Pi
allows the user to print pages of text. The Optical Character Recognition (OCR) package
installed in Raspberry Pi represents schema, modification, and segmentation before it is scanned
into a digital document and extracted for classification. Once categorised, the text is mounted on
a Raspberry Pi by a spatial speech converter (TTS engine). The output is given before playing
the audio amplifier.

5
2.1.1.3 SIGN LANGUAGE RECOGNITION SYSTEM
Aditi and N.S. (2013) proposed a system based on a static hand gesture which consists of
gestures captured using web camera processing, processing edges and peak detection and
recognition image. With the help of a webcam, an image is captured and stored in a DCR file
(Digital Camera Raw). The image is then converted from RGB (Red Green Blue) to grayscale.
Unfortunately, it was found that grayscale was less instructive for edge detection so it was
replaced with a canny edge algorithm. After detecting the edges, the Wavelet family method is
used to detect the tips of fingers. As soon as fingers are detected, the gesture is matched with
those in the database using a fuzzy set. Six alphabets are considered i.e., alphabet A, D, J, O, P,
and Q. Time recognition for the alphabet is A= 1.52, D=1.50, J=1.61, 0=1.50, p=1.57.

2.1.1.4 RECOGNITION OF INDIAN SIGN LANGUAGE IN LIVE VIDEO

Singha and Das (2013) developed a sign language recognition system which is based on static
gestures and uses stored images to recognize the gesture. They developed a model which
consisted of Eigenvalues and Eigenvectors. They used Eigenvalues weighted Euclidean distance
and developed the system based on Indian sign language. The system has four modules namely
skin colour filtering, hand cropping, extracting features and classification. 24 signs are used in
the system, each consisting of 10 samples.

LOOPHOLES IN CAMERA BASED SYSTEMS

• Sign language systems helps deaf and dumb people to communicate each other. But not
all people understand sign language. So, Communication between deaf-mute and a
normal person is still a challenging task.
• Most of the systems are online based and this does not favour users who do not and
cannot afford to have access to the internet.
• Drawback of Visual Assistance Systems for Label Detection Via Voice Output system is
that only the right hand can be used and external objects should not be present in the
background.
• Some systems are bulky.
• Video communication systems for deaf people are limited in terms of quality and
performance.

2.1.2 OTHER SYSTEMS

2.1.2.1 FULL DUPLEX COMMUNICATION SYSTEM FOR THE HEARING

IMPAIRED
Shraddha R et al. (2015) Proposed a "full duplex communication system for the hearing
impaired" A useful tool to facilitate communication between deaf and dumb people. It facilitates
two-way communication using the HMM (Hidden Markov) model to interpret gestures as voices

6
and vice versa through planned methodologies. This system overcomes the time limit of people
with speech impairments and improves behaviour. This system converts audio signals into
speech signals that can be easily comprehended by others. The speech of the normal person in
the voice form is converted into the gesture of the text and the hand.

2.1.2.2 ASSISTIVE TRANSLATOR FOR THE DEAF AND DUMB PEOPLE

"Assistive Translators for the Deaf and Dumb people" Shroke et al. (2014)
The project targets people through glove-based deaf-mute communication systems. Gloves are
designed internally with five bending sensors, touch sensors and accelerometers. For each
distinct gesture, the bending sensor develops a proportional resistance difference and the
accelerometer reads the direction of the hand. The sensor output is analogous to the digitally
converted value. The output from the sensor is analog values and it is transformed to digital. The
transformed meaning of these hand gestures is in the ARM processor which compares the input
signal with predefined voltage levels reserved in memory. According to the results, the required
output sound is produced which is saved in SPI memory with the help of the speaker.

2.1.2.3 SPEECH RECOGNITION TECHNOLOGY: APPLICATIONS & FUTURE

Communicate with normal people. Pankaj (2012) proposed “Speech Recognition Technology:
Applications & Future,” paper discussed about: Voice or speech recognition technology converts
words or phrases spoken by any person into electrical signals. These signals are transformed into
coding patterns to which meaning has been assigned. Some companies have developed a robust
system that performs as expected and sends the call to their intended destination. This
technology would have to be suitable with all software and hardware. This technology would
require the CPU (Central Process Unit) to concurrently process voice input and data access.

2.1.2.4 SMART ASSISTIVE DEVICE FOR PHYSICALLY CHALLENGED PEOPLE

Veekshita et al (2018). Voice or speech recognition technology converts words or phrases
spoken by any person into electrical signals. These signals are transformed into coding patterns
to which meaning has been assigned. Some companies have developed a robust system that
performs as expected and sends the call to their intended destination. This technology would
have to be suitable with all software and hardware. This technology would require the CPU
(Central Process Unit) to concurrently process voice input and data access.

2.1.2.5 DESIGN OF A COMMUNICATION AID FOR PHYSICALLY CHALLENGED

Suganya and Meeradevi (2015): This project paper proposes the implementation of a system
without handheld gloves and sensors and by capturing the gestures continuously and converting
them to voice and the other way around. Communication is made easier for deaf and dumb
people by using a handheld embedded device in conjunction with the hardware setup. The
effectiveness of the work is verified under the MATLAB (Matrix Laboratory) environment and

7
further in future dedicated voice output are becoming to be produced a bit like the text and thus
the gesture images captured.

2.1.3 TRAINED MODELS

During our literature review, we found different speech to text models but many were not
performing as expected. So, we decided to settle down with three models to be the best
performing out of all the models we found. These models were developed, trained and open
sourced by Mozilla (2020). The models were developed on Google’s tensor flow machine
learning framework. The models are: -

• deepspeech-gpu
• deepspeech-tflite
• deepspeech-pdm

Deepspeech-gpu
This model only works with machines that have Graphical Processing Units (GPUs) or Graphic
cards. We installed our program on a machine that had a GPU and we found that the program
worked best with this model. We analysed the model and found that it performed at a 0.9564
score rate which is about a 95% success rate. The problem we faced is that the model could not
be installed on a Raspberry Pi computer since its GPU is too weak to run the model.

Deepspeech-pdm
This model could not work on a GPU so we installed it on a machine that had a Central
Processing Unit (CPU) that is based on intel architecture. The model performed well with a score
rate of 0.9455, which is a 94% success rate. Although this model performed well. It was not
recommended by Mozilla to be installed on a Raspberry Pi, since the CPU of a Raspberry Pi is
based on ARM architecture. Nevertheless, we managed to run our program using this model on a
Raspberry. The model performed with a score rate of 0.4782. The program couldn’t recognize
the audio input properly so we did not move forward with it.

Deepspeech-tflite
This model performed the best on a Raspberry Pi since the model was trained by Mozilla to
specifically run-on ARM based architecture, which the Raspberry Pi’s CPU was built on. The
model performed with a score rate of 0.8658 which is an 87% success rate. We found that out of
all the three models this one yielded the best results on a Raspberry Pi. This is the model we
selected to run with our program. This model could not detect speech in real-time. This is the
main issue we found with the model. To overcome this challenge, we decided to train the model
again.

CONCLUSION

Per the above survey, we decided to use the deepspeech-tflite in our program and it was found
that the solutions for deaf and dumb are not contained in a single device, some systems are bulky
and expensive to use and require technical skills to operate, not forgetting that most of them are
online systems. In this proposed system, the Communication Aid for the Deaf and Mute, these
impairments are addressed into a single device.

8
CHAPTER THREE

RESEARCH METHODOLOGY/ RESEARCH

DESIGN
This part describes the selected methodology for research, data collection and analysis, project
design and implementation, testing and validation among others; including the instruments,
approaches, processes and techniques, computer hardware and software tools that will be
employed in the research study.

3.2 OVERVIEW
Our project is an experimental research, different prototypes were designed and tested by the
users and feedback from the user was used to improve the prototypes.
Using different forms of data collection such as random sampling of literature, desk review of
relevant literature, journals, and books surveys and observation of the deaf and mute, different
sources of relevant data and existing systems were reviewed. This helped us to identify the needs
and requirements of the users for the development of the Communication Aid. This also further
facilitated the design of components of the system to come up with a system block diagram. The
system was later on evaluated by making use of feedback and reports from the representative
user.

Table 1 Showing Research Objectives and Methodology

Objectives How to achieve the Objectives

To study the current communication aid Examining existing systems, literature and
systems for the deaf and mute and identify desk review.
areas that would need improvement. Carrying out online and offline surveys,
observation and focus group discussions.
To translate speech to text. Use a trained model to convert sound to
written English text
To translate text to speech. Use a library to convert written English text
to sound.
To implement then evaluate the prototypes Record the technical performance of the
and the user experience of the system prototypes and the performance of typical
users performing operations.

9
3.3 RESEARCH METHODOLOGY
Through observation research methods, the requirements for the system and the loop-holes in the
existing systems were identified and on-site observational study was conducted to evaluate the
usability of the system.

A systematic literature study was used to identify and study all relevant evidence. This included
systematic literature review which was used in finding existing evidence and systematic mapping
which was used to categorise the existing systems in order to show the gaps in the systems. The
study sampled all willing participants around Makerere and its peripheries.

3. 3.1 RESEARCH PROCESS

The following are the steps we took in order to carry out this research.

1. Literature study - At first, a literature study was made in order to gather information
about how the deaf and mute communicate with the normal people and to get an
overview of the functionalities of communication Aids for the deaf and mute. The choice
of sources was mainly articles on the subject as well as relevant papers in databases such
as International Journal of Innovative Technology and Exploring Engineering,
ScienceDirect and ResearchGate. The search words among others were: communication
aid for the deaf and mute, communication aid for deaf and normal people,
communication between impaired people and normal people, communication aid for the
mute. The credibility of these sources was checked through a comparison of other
materials and inspected to see if there was any biased or outdated information on the
sites. The primary results of the literature study are presented in Chapter 2 (literature
review).
2. Determine functionality - Secondly, a decision on the required functionalities of the
system was made. A list of functions was produced from the reviewed systems. These
functions were the foundation for the prototype.
3. Select usability evaluation parameters - In order to evaluate the usability of these
functions, evaluation parameters were selected. The selected parameters were the ISO
9241-11:2018 standard parameters for usability: Efficiency, Effectiveness, and
Satisfaction.
4. Create a prototype - The user interfaces of the prototype have been created following
Schneiderman’s eight golden rules and Nielsen’s ten heuristics. And the overall design of
the system was inspired from the reviewed systems in the literature review.
5. Define a test group - Thereafter, a test group was constructed. The selection of this
group is described in Section 3.3.2.
6. Choose evaluation methods - After the test group was constructed, the usability of the
prototype was evaluated through user-based evaluation. The usability evaluation methods
we chose were post-test interviews and a combination of performance measurement and

10
 questionnaire. The questionnaire was used to collect performance data regarding user
satisfaction. We observed the test subjects performing each task, having them comment
during testing and measured a set of measurements. For example, the time and speed to
produce output and accuracy of the output. At the end of the test, the subjects rated their
satisfaction of the interaction with the prototype.
7. Design questionnaire and interview - Based on our research objectives we developed
questions for our questionnaire. The interview was to be conducted individually with
each test subject in a quiet setting.
8. Capture data - To capture data, participants' answers, expressions and comments were
noted on paper during interviews. Performance data such as time, speed and accuracy
were noted on paper. The user interactions with the prototype as well as errors made were
recorded. The questionnaire was filled out online by the participants using kobo collect.
9. Analyse data - The data from the study were both quantitative and qualitative.
Qualitative data (such as the content of comments and expressions and the types of errors
made) were more difficult to analyse than the quantitative data. The number of negative,
neutral, and positive comments were counted. The median value of the performance
measurements was calculated. The quantitative data was used to measure the level of
usability of the prototype and the participants’ opinion in regard to communication aids in
general. While the qualitative data was used to find areas of improvement.
10. Critique UI to suggest improvements - After the data was analysed, suggested
improvements were based on the qualitative data as well as the quantitative data.
11. Interpret and report - make sense of the numerical and qualitative data that has been
collected, analysed and presented. Record our findings in a report, and associate
interpretations to justify our conclusions and recommendations.

3.3.2 DATA COLLECTION AND ANALYSIS

The study used both primary and secondary data in order to accomplish the research objectives
and help generate conclusions and recommendations.
1. Primary Sources
The study adopted a questionnaire and an interview guide as instruments for data collection
in a joint qualitative and quantitative approach. For example;
• To study the current communication aid systems for the deaf and mute, the team made a
survey using online data collection tools (Kobo Collect) which focused on evaluating the
current systems for the deaf.
• Focus group discussions facilitated the interviewing of the deaf and mute. In this case, the
team designed a small questionnaire with both open ended and closed questions so as to
capture all the relevant information.
• We also carried out key informant interviews where a sample of leaders of persons with
disabilities were interviewed to get a clear picture of the existing systems and the
loopholes in them.

11
 • Through the observation method, we were able to observe how the deaf and dumb
communicate with each and how they are interacting with the existing communication
aids. The findings were analysed and reported.

2. Secondary Data Sources

Secondary Data Sources may be referred to as data that is not originally gathered by the
study, and help in some way to arrive at a conclusion for the study. Secondary data sources
according to Sekaran (2003) are derived from data that is already in existence. Secondary
data for this study was acquired from a variety of online databases of journals, books, and
other such sources. Secondary data was very instrumental in gathering requirements of the
system. For example,
• Desk review of existing systems was carried out, where each system evaluated so as to
identify the loopholes, and also evaluate the need for the proposed system.
• Qualitative data was collected through literature reviews; reading the necessary
documentations, journals, books and articles on the internet. The qualitative data
generated was transcribed, reported and analysed. Models will be generated from
analysing data to extract insights that support decision-making.
• To analyse the areas that require improvement, the collected data was analysed using
python for quantitative data and advanced excel for qualitative data. Models were
generated to come up with an improved system.
• Post-campaign surveys were conducted after developing the product to see how effective
it is to the target population.
• Experimental methods were also used to assess whether the system met the required
functions.

3.2.3 DATA ANALYSIS

Quantitative data analysis was done with the help of IBM SPSS Statistics (SPSS) software, kobo
collect tool and Microsoft Office Excel 2019. Quantitative data collected was first coded and
entered into IBM SPSS to pave the way for easy analysis. Descriptive aspects of the findings
were presented with the help of tables, pie charts and graphs alongside other descriptive
statistical indicators.
Qualitative data was manually coded by the researcher. This significantly reduced the amount
of data available for analysis. Qualitative data was analysed alongside quantitative data in a
concurrent triangulation approach to mixed methodologies.

12
3.4 SYSTEM DEVELOPMENT METHODOLOGY
An embedded system design process, in line with the Agile methodology, was used in the
development of the proposed communication aid system. This method allowed team members to
involve users and make continual adjustments and improvements as we develop the system.
The team leader acted as a project manager to oversee the system design process and ensured
that work was meeting the product development timeline. This ensured that the project is
completed in a timely manner.

The following steps were followed under this system design process.
• Determine overall requirements. Based on the data collected through observation, surveys
and random sampling, data was analysed using SPSS (Statistical Package for The Social
Sciences) and Microsoft Excel. It was used to determine the requirements of the system
such as size, weight and cost limits as well as the specific hardware the product will use.
It also helped decide whether or not the product needed a user display and if needed, the
kind of display and graphical user interface that would be implemented.
• Document detailed technical specifications. The team created this report detailing the
technical specifications of the system which also include functions the system must
complete, environmental conditions and manufacturing requirements, among others.
• Develop prototypes. Using Proteus and Microsoft Visio, we designed basic prototypes to
assess the hardware, identify necessary components and determine how they would work
together. These prototypes were used during system analysis, optimising, debugging, and
testing.
• Design system architecture. Raspberry Pi and other peripheral devices such as
microphones, speakers and all other modules were used to develop the system
architecture.
• Select the operating system. We decided to use Raspberry Pi OS (Raspbian) as a real-
time operating system.
• Choose the development platform. We used Python as the programming language to
develop the AI for the communication aid. This decision was based on the analysis of the
system requirements, strength, ease of development, expressive power, longevity,
runtime efficiency, ease of maintenance, module availability, connectivity support and
developer community.
• We used GitHub as our Version Control System to manage the project.
• Emulators such as Proteus were used to emulate the communication aid so as to test and
validate the system prototype.
• After the development process, the team tested the system using observation methods and
a development survey was conducted on a random sample of users. Feedback was
collected, analysed and evaluated so as to make the necessary improvements to the
system.

13
 • Ongoing maintenance and updates will be carried out as the team continually monitors
how the system works, make necessary adjustments and provide updates to fix problems.

CONCLUSION
In order to achieve our research objectives, we chose research methods that are well known and
scientifically used in the field of Information System Development and Computer Science. To
get valid results from the study, it was important to not interrupt or affect the participants in any
manner that would coerce their answers. Ethical principles were observed throughout the course
of this research.
The sample size (number of participants) may affect the validity of the results. Since we used
several independent test subjects for the evaluation of our prototype, we estimate that our study
is reliable.
Through the data collected and analysed during the study, we were able to gather the different
requirements of the system, analyse the system’s stakeholder’s opinion regarding benefits and
obstacles of the Communication Aid, and assess the usability of our prototype.

14
CHAPTER FOUR

RESULTS/ FINDINGS
4.1DATA COLLECTION
We conducted a survey using questionnaire in order to examine the existing means of
communication for the deaf and mute and their challenges and find out the need for our proposed
system. The results were used to draw function requirements for our proposed system.

This survey was carried out online also to examine the communication challenges among the
deaf and how they are able to cop up despite the above challenges, Student leaders for persons
with Disabilities from Makerere and Kyambogo universities were reached to who helped us
reach to the deaf around Makerere and Kyambogo Universities. Since the study was around these
two universities, most of the respondents were students or old students from these two
institutions or the surrounding institutions.
Due to lack of funds to hire a sign language interpreter, a WhatsApp group was created where I
briefed the deaf about the project, and also got feedback from them. This as well eased
distribution of the online Survey tools so as to ease the data collection process.
However, out of the 30 deaf in the WhatsApp group, only 20 managed to fill up the survey since
others were offline and couldn’t fill the survey in time.

The respondents were distributed as follows based on their gender:

60% of the respondents identified themselves as Male, whereas the 40% as Female.

Figure 1 pie chart showing gender Distribution

15
From the people interviewed, 80% were between 19 to 25, 10% were between 26 to 30 whereas
the remaining 10% were above 30 years.

Figure 2 Bar Graph showing age distribution

Figure 3 Bar Graph showing age distribution by gender

16
Since the survey was spearheaded by student leaders of the persons with Disabilities, 10% of the
people interviewed were students in Advanced Level (A ‘level), 70% were Undergraduate
students whereas the 20% were students in Vocational institutions.

Figure 4 Bar Graph showing Current Level of education

School / College/ Institute

65% percent of the Undergraduate students were from Makerere University, 5% from Kyambogo
University, 10% from Makerere Business School and the rest from Buganda Royal Institute.

Figure 5 Bar Graph showing School or institute attended by interviewee

17
District
Since the study was carried out around Makerere which is located in Kampala District, we
needed to find out the home districts for the deaf we interviewed. From the survey, 50% were
from Kampala district whereas the 50% were equally distributed in the districts of Lira, Gomba,
Pakwach, Jinja and Amolatar.

Figure 6 Bar Graph showing district distribution

From the survey respondents, 60% have no care takers, and only 40% have care takers. It was
also noted that all those who have care takers, it’s the Parents acting as their care takers, thus no
other external care taker since the parents are low-income earners who cannot afford hiring a
care taker.

Figure 7 Pie chart for Care taker

Level of hearing problem

18
80% of the people who filled up the survey were completely deaf but could talk (Deaf but not
mute), 10% were completely deaf and could not talk (Deaf and Mute) whereas the 10% had
hearing impairment (Could slightly hear).

Figure 8 Pie chart showing level of hearing

Means of Communication
60% of the deaf who filled up the survey use sign language interpreters, whereas the 40% find
their own way of communicating. In this survey, none of the deaf interviewed use a
communication device.

Figure 9 Pie chart showing means of communication

19
For those who use sign language interpreters as their means of communication, 50% pay the
interpreter on Monthly basis, 17% pay them on daily basis whereas the 33% pay them per
semester.

Figure 10 Pie chart showing Basis of Payment

Need for a communication aid device
100% of the respondents expressed interest and need for a communication aid device that will
help them communicate freely with others with privacy, and also minimizing the costs.

Figure 11 Pie chart showing need for our proposed system

20
 4.2 PROPOSED SYSTEM
Based on the survey we conducted, we propose a communication Aid for the deaf and dumb that
can translate text to speech (for the mute or dumb) and speech to text (for the deaf) to help them
in their communication.

Figure 12 System block diagram

Figure 12 shows the architectural diagram of the proposed system. The system has microphone,
Display/screen which might be embedded with the case or standalone monitor of the raspberry
pi, speaker and SD card.

4.2.1Hardware components

1. Raspberry Pi 4 model B (2GB RAM)

Raspberry Pi is a credit-card sized computer that connects to a computer monitor or TV and uses
input devices like keyboard, speaker, microphone and mouse.

21
Raspberry Pi 4 Model B is the latest product in the popular Raspberry Pi range of computers. It
offers ground-breaking increases in processor speed, multimedia performance, memory, and
connectivity compared to the prior-generation Raspberry Pi 3 Model B+, while retaining
backwards compatibility and similar power consumption. For the end user, Raspberry Pi 4
Model B provides desktop performance comparable to entry-level x86 PC systems.

Figure 13 Raspberry pi 4 model B

This product’s key features (figure 13) include a high-performance 64-bit quad-core processor,
dual-display support at resolutions up to 4K via a pair of micro-HDMI ports, hardware video
decode at up to 4Kp60, up to 4GB of RAM, dual-band 2.4/5.0 GHz wireless LAN, Bluetooth
5.0, Gigabit Ethernet, USB 3.0, and PoE capability (via a separate PoE HAT add-on).

The dual-band wireless LAN and Bluetooth have modular compliance certification, allowing the
board to be designed into end products with significantly reduced compliance testing, improving
both cost and time to market.

22
Specification

Table 2 Specifications of the raspberry pi 4 model B

Processor Broadcom BCM2711, quad-core Cortex-A72

(ARM v8) 64-bit SoC @1.5GHz
Memory 1GB, 2GB or 4GB LPDDR4 (depending on
model)

Connectivity 2.4 GHz and 5.0 GHz IEEE 802.11b/g/n/ac

wireless LAN, Bluetooth 5.0, BLE
Gigabit Ethernet
2 × USB 3.0 ports
2 × USB 2.0 ports.

GPIO Standard 40-pin GPIO header (fully

backwards-compatible with previous boards)
Video & sound 2 × micro-HDMI ports (up to 4Kp60
supported)
2-lane MIPI DSI display port
2-lane MIPI CSI camera port
4-pole stereo audio and composite video port

Multimedia H.265 (4Kp60 decode);

H.264 (1080p60 decode, 1080p30 encode);
OpenGL ES, 3.0 graphics

SD card support Micro SD card slot for loading operating

system and data storage

Input power 5V DC via USB-C connector (minimum 3A)

5V DC via GPIO header (minimum 3A)
Power over Ethernet (PoE)–enabled (requires
separate PoE HAT)

Environment Operating temperature 0–50ºC

OS (Operating System) Boots from SD card and running a version of
Linux Raspbian OS.

2. SD card
It is a type of memory card typically used in digital cameras and other portable devices.
SD card is an ultra-small flash memory card designed to provide high-capability memory
in a small size.

23
Our system is using a 32 GB SD card and this is where the operating system and the code for the
system are stored.

3. Speaker

Figure 14 speaker connect by Bluetooth Figure 15 Speaker connected by cable

The speaker is connected to the raspberry PI either by a cable or by Bluetooth, which will display
the output in the form of voice. The speaker used in the proposed system is a basic model which
is used only for audio purposes.

4. Touch screen case

Figure 16 Touch Screen case for Raspberry pi

Figure 16 shows how the Raspberry pi is embedded into the touch screen (3.5-inch LCD) case
for display of text and providing an interface for typing text which is later converted to speech.
In case one cannot buy a touch screen case, he/she can use a standalone monitor for display,
keyboard for input and mouse.

24
 5. Microphone

Figure 17 showing microphone connected to Raspberry Pi

The microphone is connected to the Raspberry pi for voice input using a cable as shown in figure
17.

4.2.2 Software Requirements

Raspberry Pi needs an operating system to work. This is it. Raspberry Pi OS (previously called
Raspbian) is the official supported operating system. This was done with help of Raspberry Pi
Imager, Raspberry pi imager is the quick and easy way to install Raspberry Pi OS and other
operating systems to a microSD card, ready to use with the Raspberry Pi.
These are the steps we took to install the operating system
• Download and install Raspberry Pi Imager to a computer with an SD card reader.
• Put the SD card you'll use with your Raspberry Pi into the reader and run Raspberry Pi
Imager.

The system was coded in python version 3.7 and trained machine learning model which converts
speech to text and text to speech. We used Python because python is a high-level, interpreted,
interactive and object-oriented scripting language. Python is designed to be highly readable. It
uses English keywords frequently where as other languages use punctuation, and it has fewer
syntactical constructions than other languages.

In this project we used visual studio code as our IDE because it had all the tools we needed to
develop the software.

25
4.2.3 Machine learning model
During our survey, we found different speech to text models but many were not performing as
expected. So, we decided to settle down with three models to be the best performing out of all
the models we found. These models were developed, trained and open sourced by Mozilla. The
models were developed on Google’s tensor flow machine learning framework. The models are: -

• deepspeech-gpu
• deepspeech-tflite
• deepspeech-pdm

Deepspeech-gpu
This model only works with machines that have Graphical Processing Units (GPUs)/ Graphic
cards. We installed our program on a machine that had a GPU and we found that the program
worked best with this model. We analysed the model and found that it performed at a 0.9564
score rate which is about a 95% success rate.

Limitations of Deepspeech-gpu.
The problem we faced is that the model couldn’t be installed on a Raspberry Pi computer since
its GPU is too weak to run the model.

Deepspeech-pdm
This model couldn’t work on a GPU so we installed it on a machine that had a Central
Processing Unit (CPU) that is based on intel architecture. The model performed well with a score
rate of 0.9455, which is a 94% success rate.

Limitations of Deepspeech-pdm
Although this model performed well. It was not recommended by Mozilla to be installed on a
Raspberry Pi, since the CPU of a Raspberry Pi is based on ARM architecture. Nevertheless, we
managed to run our program using this model on a Raspberry. The model performed with a score
rate of 0.4782. The program couldn’t recognize the audio input properly so we did not move
forward with it.

Deepspeech-tflite
This model performed the best on a Raspberry Pi since the model was trained by Mozilla to
specifically run-on ARM based architecture, which the Raspberry Pi’s CPU was built on. The
model performed with a score rate of 0.8658 which is an 87% success rate. We found that out of
all the three models this one yielded the best results on a Raspberry Pi. This is the model we
selected to run with our program.

Limitations of Deepspeech-tflite
This model couldn’t detect speech in real-time. This is the main issue we found with the model.
To overcome this challenge, we decided to train the model again, but this did not yield good
results because of the limited time we had to train the model.
Despite its challenges, we decided to use the deepspeech-tflite in our program because it
compatible with our raspberry Pi.

26
Re training of the deepspeech-tflite model

We installed the machine learning model on the Raspberry Pi and found that the model didn’t
work as expected. The reason is because the model was trained on audio datasets with accents
from individuals of American ethnicity.
We retrained the model with our own accents as well as other recorded accents from volunteers.
This increased the accuracy of the model by 5%. This is because the model was being trained on
hardware with insufficient resources like Random Access Memory(RAM).

4.2.4 Brief view of our system output

Figure 18 Output for text to speech (for the mute)

Figure 18 shows what happens when the user (the mute) types the message and at the same time
the system keeps on listening to voice input in real time and speed. The message is sent to the
speakers as audio.

27
 Figure 19 output for speech to text (for the deaf)
Figure 19 shows what happens when the user (the dumb) speaks the message and at the same
time the system allows text input. The message is sent to the monitor or screen as text.

4.3 CONCLUSION
The source code for this prototype plus additional files have been stored and shared through
GitHub. All files can be found at https://github.com/josephkabba/seech-recognition

Following the implementation of the prototype we conducted a usability study of the

prototype. Our main parameters of evaluation were time, accuracy and speed. We evaluated the
time it took for the system to produce audio and display text and how accurate are the words
produced by the system. We also evaluated how fast the system is in producing output like audio
and text.
Below is a summary of the results from the study.

Parameters General overview

Time 5seconds

Accuracy 87% accurate

Speed Moderately fast

Table 3 Performance Results

28
CHAPTER FIVE

DISCUSSION OF RESULTS/ FINDINGS

5.1 RESULTS FROM THE SURVEY
From the survey, the minimum monthly payment given to sign language interpreters is 333,000
Uganda shillings (85 US Dollars), the maximum monthly payment is 1,500,000 Uganda shillings
(385 US Dollars) whereas the average monthly payment to sign language interpreters is 710,000
Uganda shillings (182 US Dollars). We also found out problems associated with sign language
interpreters which the proposed system is going to solve.

The major problems addressed by the respondents which are associated with Sign language
Interpreters include high cost of paying them since they are paid huge amounts of money, they
are very rare and hard to get, additional costs paid to sign language interpreters as well as lack of
privacy.

On the other hand, the problems faced by the respondents during communication
(Communication challenges) as addressed in the survey include;
Attitude and Stigmatism, many people do not understand sign language especially the hearing
ones, failure to get what others are talking, Lack of privacy especially when using an interpreter
and difficulty in understanding the communication because of no interpreter.

While at school, they face a challenge of discrimination by fellow students, failure to understand
what is being taught especially those who have no sign language interpreter, and to those who
have sign language interpreters said that sometimes interpreters miss-interpret information.
Failure to communicate with staff members is also a big issue among the deaf while trying their
best to acquire education.

Need for a communication aid device

As shown by figure 11, 100% of the respondents expressed interest and need for a
communication aid device that will help them communicate freely with others with privacy, and
also minimizing the costs.

5.1.1 CHALLENGES FACED DURING RESEARCH

• Training the models is financially and computationally very expensive and take long
time.
• The model was trained on datasets with audio of individuals from American ethnicity.
• Communicating with the deaf and mute was a challenge.

29
 • Very few people were willing to take part in the survey.

5.2 LIMITATION OF THE SYSTEM

• The system cannot efficiently recognise speech in real-time.
• The system cannot efficiently recognise speech from individuals of African ethnicity
since the model was trained on datasets with audio of individuals from American
ethnicity.
• The system only recognises English language.
• The system cannot efficiently recognise speech in noisy environments.
• The system does not filter out noise and the message of the user. The system captures
every sound in the environment and thereby affecting the accuracy of the output.

5.3 FUTURE WORK

• More work needs to be done in fine-tuning the system to recognise local languages.
• Noise cancellation and normalization techniques are need in training the system to reduce
the channel and the environment effects because noise affects the accuracy of the system.
• The system should be retrained more using African accents.
• More research should be done on optimization of the speed of the system so that it can
produce output in the fastest way possible.

30
CHAPTER SIX

CONCLUSION
In conclusion, the team has worked around the clock to have a prototype to a solution
proposition that would solve a real-world problem. And it has been quite a journey with lots
of lessons and challenges. But notably, we were able to put in practice many of the concepts
we had learnt throughout our stay at Makerere University.

The main purpose of this project is to aid the communication of the deaf and dumb so that they
can easily communicate with each other and with a normal person.

This system is portable and less costly as compared to the existing systems. It works offline and
thus it can be used in remote areas where there is no internet connection. The system can be used
by the deaf, mute and a normal person since it translates text to speech and speech to text. It is
therefore against such a background that the need to hire sign interpreters is erased and
communication characterized by privacy accelerated.

The system has been carefully developed through thorough research and a complex degree of
team work accompanied by close supervision and guidance by our supervisor. It is at this point
therefore that it ought to be noted that the system perfectly translates text to speech and speech to
text however there exists room for improvement in regards to further research and study
accordingly.

31
CHAPTER SEVEN

RECOMMENDATIONS

We recommend further research in training the model to run in real-time as well as recognising
speech from different ethnicities and accents.

Noise is really a big deal; it can increase the error rate of speech identification system. So, use of
noise cancellation and normalization techniques to reduce the channel and the environment
effects is recommended. Also, voice activity detection should be done. All of these can improve
the recognition accuracy.

We also recommend further research to be done on optimization of the speed of the system so
that it can produce output in the fastest way possible.

32
REFERENCES
Center for Human Rights, U. of P. (1995) Repository on Disability Rights in Africa - UN
Documents. Available at: https://rodra.co.za/index.php/un-docs-uganda (Accessed: 10 May 2022).

Disability and Health Overview | CDC (Page last reviewed: September 16, 2020). Available at:
https://www.cdc.gov/ncbddd/disabilityandhealth/disability.html (Accessed: 10 May 2022).

Ministry of Gender Labor and Social Development (MoGLSD) (2016) Sector Development Plans
– National Planning Authority, Uganda Govt. Available at: http://www.npa.go.ug/development-
plans/sector-development-plans/ (Accessed: 10 May 2022).

National Association of Sign Language Interpreters (UNASLI) website - Google Search (no date).
Available at:
https://www.google.com/search?q=National+Association+of+Sign+Language+Interpreters+%28
UNASLI%29+website&sxsrf=ALiCzsZvVmnbHYRiwGFoN6z7bk9lIbitQ%3A1651083613757
&ei=XYlpYr7mLc63kgXQtq34BQ&ved=0ahUKEwjiKvi7bT3AhXOm6QKHVBbC18Q4dUDC
A4&uact=5&oq=National+Association+of+Sign+Language+Interpreters+%28UNASLI%29+we
bsite&gs_lcp=Cgdnd3Mtd2l6EAMyBAghEAo6BwgAEEcQsAM6BAgjECdKBAhBGABKBAh
GGABQ1hJYzVFghFZoAnABeACAAZIGiAHuIJIBCTMtMS4wLjUuMZgBAKABAcgBB8A
BAQ&sclient=gws-wiz (Accessed: 27 April 2022).

NCD (2019) National Council for Disability | Opportunity for Equity, published. Available at:
https://www.ncd.go.ug/ (Accessed: 10 May 2022).

NDP (no date) Sector Development Plans – National Planning Authority. Available at:
http://www.npa.go.ug/development-plans/sector-development-plans/ (Accessed: 10 May 2022).

Translating, interpreting, Uganda salaries - Paylab.com (no date). Available at:

https://www.paylab.com/ug/salaryinfo/translating-interpreting (Accessed: 27 April 2022).

Uganda (2016) OHCHR | CRPD/C/UGA/CO/1: Concluding observations on the initial report of

Uganda, published. Available at: https://www.ohchr.org/en/documents/concluding-
observations/crpdcugaco1-concluding-observations-initial-report-uganda (Accessed: 10 May
2022).

Uganda Govt (2015) Second National Development PLAN (NDPII) 2015/16 – 2019/20 | Ministry
of Health Knowledge Management Portal, Name of publisher: Uganda Govt Place of publication:
Kampala Category: Work Plans. Available at: http://library.health.go.ug/publications/work-
plans/second-national-development-plan-ndpii-201516-–-201920 (Accessed: 10 May 2022).

WHO (2001) International Classification of Functioning, Disability and Health (ICF), published.
Available at: https://www.who.int/classifications/international-classification-of-functioning-
disability-and-health (Accessed: 10 May 2022).

33
WHO (2005) World Health Statistics, WHO. Available at:
https://www.who.int/data/gho/publications/world-health-statistics (Accessed: 10 May 2022).

WHO team (2010) The World Health Report 2010, WHO. Available at:
https://www.who.int/publications/i/item/9789241564021 (Accessed: 10 May 2022).

Government of Uganda (2016): National Population and Housing Census Report 2014, Kampala,
Uganda Bureau of Statistics (http://www.ubos.org). (UBOS (2016) Uganda Demographic Health
Survey 2016; UBOS (2016) National Population and Housing Census 2014; UBOS (2011) Uganda
Demographic Health Survey 2009/2010 and; UBOS (2006) Uganda Demographic Health Survey
2006).

UNDP. 2016. Human Development Report 2016: Human Development for Everyone. New York.
http://hdr.undp.org/en/content/human-development-report-2016

US Department of Health and Human Services (2005). The Surgeon General’s Call to Action to
Improve the Health and Wellness of Persons with Disabilities external icon. Washington, DC: US
Department of Health and Human Services, Office of the Surgeon, from
http://www.ncbi.nlm.nih.gov/books/NBK44667/

National Council for Disabilities (AMENDMENT) ACT 6, 2013. From

https://www.ncd.go.ug/download/file/fid/378.

https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-
withdisabilities.html (retrieved on 5th March 2019).

National Council for Disability: Disability Status Report - Uganda, 2019. from https://afri-
can.org/wp-content/uploads/2019/08/Disability-report-2019-p-1.pdf .

final evaluation report of the “Improved knowledge and awareness of health, HIV, education
rights and choices for deaf people in Uganda” project implemented in 2015-2018 in Greater
Masaka (Central Uganda) and Arua (Northern Uganda): Daft-2110-Signal-Evaluation- Report,
form https://signhealthuganda.org/wp-content/uploads/2018/07/Daft-2110-Signal- Evaluation-
Report.pdf .

Mozilla. (2020). DeepSpeech Model — Mozilla DeepSpeech 0.9.3 documentation. 2020

DeepSpeech Authors Revision Bfccca32.
https://deepspeech.readthedocs.io/en/r0.9/DeepSpeech.html

Research Process As reported by Uma Sekaran (2003), through research... | Download Scientific
Diagram. (n.d.). Retrieved September 27, 2022, from
https://www.researchgate.net/figure/Research-Process-As-reported-by-Uma-Sekaran-2003-
through-research-design-improvement_fig1_270338273

K. Pieri, S. V. G. Cobb, “Mobile app communication aid for Cypriot deaf people,” Journal of
Enabling Technologies, 2019, doi:10.1108/JET-12-2018-0058

34
Asha G. Hagargund, Sharsha Vanria Thota, Mitadru Bera, Eram Fatima Shaik, “Image to Speech
Conversion for Visually Impaired”, International Journal of Latest Research in Engineering and
Technology, Volume 03, Issue 06, June 2017.

Chucai Yi, Student Member, IEEE , Yingli Tian, Senior Member, IEEE , and Aries Arditi
“Portable Camera-Based Assistive Text and Product Label Reading From Hand-Held Objects for
Blind Persons” 2013 IEEE.

K. Lakshmi, Mr. T. Chandra Sekhar Rao, “Design and Implantation Of Text To Speech
Conversion Using Raspberry PI”, International Journal of Innovative Technology and Research,
Volume No. 4, Issue No. 6, October-November, 2016.

Veekshita R R, Meghana R, Varsha Iyengar G, Thejaswini B R, Latha M, 2018, Smart Assistive

Device for Physically Challenged People, INTERNATIONAL JOURNAL OF ENGINEERING
RESEARCH & TECHNOLOGY (IJERT) NCESC – 2018 (Volume 6 – Issue 13).

Joyeeta Singha, and Karen Das, "Recognition of Indian Sign Language in Live video”,
International Journal of Computer Applications”, Vol.70, No.19, pp 17-22, May 2013.

R. Suganya and T. Meeradevi, "Design of a communication aid for physically challenged," 2015
2nd International Conference on Electronics and Communication Systems (ICECS), 2015, pp.
818-822, doi: 10.1109/ECS.2015.7125026.

Aditi Kalsh and N.S. Garewa, “Sign Language Recognition System,”,International Journal of
Computational Engineering Research,Vol, 03 ,Issue, 6,June,2013.

Martin Goldberg CUNY Graduate Center New York, NY. USA for “Assisting Visually Impaired
People with Mobility through Technology in the Age of Context” in 2015.

Nagaraja L., Nagarjun RS, Nishanth M Anand, Nithin D, Veena S Murthy, “Vision based Text
Recognition using Raspberry Pi”, International Journal of Computer Applications, National
Conference on Power Systems & Industrial Automation.

Pankaj Pathak Dept. of CSE Shri Vaishnav Institute of Computer science Indore, India,” Speech
Recognition Technology: Applications & Future,” International Journal of Advanced Research in
Computer Science Vol. 1, No.4, Nov-Dec 2012.

Rithika H., B. Nithya Santhoshi, “Image Text To Speech Conversion In The Desired Language
By Translating With Raspberry Pi”.

S.B.Shroke, Nandar Dehpande, Prashant Deshmukh, Sanjay Kumar

Mathapati. ”Assistive Transulator For Deaf And Dumb People”-in July-2014.

Shraddha R. Ghorpade, Surendra K. Waghamare, “Full Duplex Communication System for

Deaf & Dumb People,” International Journal of Emerging Technology and Advanced

35
Engineering (IJETAE), Volume 5, Issue 5, May 2015, ISSN 2250-2459.
Vasanthi.G and Ramesh Babu.Y Department of ECE, DMI College of Engineering, Chennai,
India.“Vision Based Assistive System for Label Detection with Voice Output”Jan-2014.

http://youtu.be/QUta4f_87E 5

http://www.youtube.com/playlist?list=PL87CgTAGz2yZHV2FghkkivoRwfy-KErG
http://nevonprojects.com
http://youtube/CuvoEQGp8rA8.
https://developer.mbed.org/users/edodm85/notebook/HC-05-bluetooth/
http://www.raspberrypi.org/archieves/3195
https://play.google.com/store/apps/details?id=com.app.control&h1=en
https://play.google.com/store/apps/details?id=robots.simplelabs.amr_voice&h1=en

36
APPENDICES
APPENDIX A

IMPLEMENTATION BUDGET

Unit
No. Item/Activity Quantity Price Price

Raspberry Pi 4 model B 2GB with screen. This is the

1 microcontroller to be used with a display screen and casing 1 900,000 900,000

Microphone. The microphone is to be used for speech recognition.

2 1 25,000 25,000

Memory card 32GB

3 1 40,000 40,000

Speaker. This is for outputting sound or speech

3 1 50,000 50,000

Key Informant Interviews. During preliminary research, we

interviewed some persons with disabilities, particularly deaf. This
is important because it will help us get first hand information about
their communication challenges.
4 250,000

TOTAL
1,265,000

37
APPENDIX B
QUESTIONNAIRE

38
10/5/22, 12:27 PM COMMUNICATION AID FOR THE DEAF AND PEOPLE WITH HEARING IMPAIRMENT

COMMUNICATION AID FOR THE DEAF AND PEOPLE WITH

HEARING IMPAIRMENT

This survey is to be used to determine the need for a communication aid

device for the deaf which will be used as a substitute for Sign language
interpreters. Information provided will be secure and only to be used for this
particular project. Thanks.

» Demography Information

Name
surname and first name

Age Bracket

15-18

19-25

26-30

31-above

Gender

Male

Female

Other

What is your current education level?

Primary Level

O'Level

A'level

Undergraduate

Working class

Vocational / Tertiary institution

School / College/ Institute

Location (District)

https://kobo.humanitarianresponse.info/#/forms/ahYbEnMTwfCCw5bnZBPTcQ/landing 1/5
10/5/22, 12:27 PM COMMUNICATION AID FOR THE DEAF AND PEOPLE WITH HEARING IMPAIRMENT

Do you have a care taker?

Yes

No

Who is the care taker?

Parents

Relative

Other

Specify

What is the income status of your guardian?

Low income Earner

Middle income Earner

High income Earner

How would you grade your income status?

Low Income Earner

Middle Income Earner

High Income Earner

Unemployed

Your Contact (Email or phone number)

Level of hearing problem

Cannot hear at all (Deaf but can talk)

Has hearing impairment (can slightly hear)

Deaf and mute (cannot hear and talk)

Mute (Can hear but cannot talk)

Other

Specify

» Assessing the ability to communicate

https://kobo.humanitarianresponse.info/#/forms/ahYbEnMTwfCCw5bnZBPTcQ/landing 2/5
10/5/22, 12:27 PM COMMUNICATION AID FOR THE DEAF AND PEOPLE WITH HEARING IMPAIRMENT

What means of communication do you currently use for communication?

Sign Language Interpreter

Communication Aid device

I find my own way of communicating with others

Others

Specify

Which kind of device is it?

Where did you get it from?

How much did that device cost?

Do you think there are future health implications associated with using the device?

Yes

No

What do you think are future the health implications of the device?

On which basis is he/she paid?

Daily

Weekly

Monthly

Other

Specify

How much do you pay him/her?

https://kobo.humanitarianresponse.info/#/forms/ahYbEnMTwfCCw5bnZBPTcQ/landing 3/5
10/5/22, 12:27 PM COMMUNICATION AID FOR THE DEAF AND PEOPLE WITH HEARING IMPAIRMENT

What problems are associated with sign language interpreters?

Lack of privacy

High cost of paying them

They are rare and therefore hard to get

Additional costs payed to them (e.g transport, lunch)

None

Other

Specify

What problems do you face when communicating with others?

Are you interested in a communication aid system which will work the same way as a sign language interpreter?

Yes

No

How best would you prefer to access the device?

I can manage to buy it

I would like to get it through donation

I can pay for it through installments

Other

Specify

How much can you afford to pay for the device?

Why are you not interested in the communication aid?

Are you currently working?

Yes

No

Still studying

https://kobo.humanitarianresponse.info/#/forms/ahYbEnMTwfCCw5bnZBPTcQ/landing 4/5
10/5/22, 12:27 PM COMMUNICATION AID FOR THE DEAF AND PEOPLE WITH HEARING IMPAIRMENT

How do you communicate at the work place?

I use a sign language interpreter

There is a communication aid device at the work place

I have my own communication aid which i use

I can use signs to my workmates

Others

Specify

What communication challenges do you face at the work place?

Which communication challenges did you /do you face at school?

How do you / did you cop up with education despite the communication challenges?

In case of any additional comments or problems you face, feel free to provide them

https://kobo.humanitarianresponse.info/#/forms/ahYbEnMTwfCCw5bnZBPTcQ/landing 5/5