2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS)

HAND GESTURE RECOGNITION AND

VOICE CONVERSION SYSTEM FOR DUMP
PEOPLE
Vigneshwaran S1 ShifaFathimaM2 VijaySagarV3
Electronics and Communication Electronics and Communication Electronics and Communication
Engineering Engineering Engineering
SNS College of Technology SNS College of Technology SNS College of Technology
Coimbatore, India Coimbatore, India Coimbatore, India
vigneshwaransd@gmail.com shifaece15@gmail.com vijaysagar061997@gmail.com

Sree Arshika.R
Electronics and Communication
fbgbhbhEngineering
SNS College of Technology
dsfgfdCoimbatore, India
rvarshika160@gmail.com

ABSTRACT:Sign language plays a major role for
dumppeople to communicate with normal people. It is very
difficult for mute people to convey their message to normal
people. Since normal people are not trained on hand sign
language. In emergency time conveying their message is
very difficult. So the solution for this problem is to convert
the sign language into human hearing voice. There are two
major techniques available to detect hand motion or gesture
such as vision and non-vision technique and convert the
detected information into voice through raspberry pi. In
vision based technique camera will be used for gesture
detection and non-vision based technique sensors are used.
In this project non-vision based technique will be used.Most
of the dumb people are deaf also. So the normal people’s
voice can be converted into their sign language. In an
emergency situation the message will automatically send to
their relation or friends.
development of vision-based sign recognition systems
through worldwide. Vision-based gesture recognition
systems can be divided into direct and indirect methods.
In earlier days for recognizing hand motion, vision based
technique is used. But in this method the environmental
effect in the recognized image is high. And another
disadvantage is they have to show their hands to in front
of the camera.Here flex sensor is used for detecting the
hand motion and convert it into voice using
microcontroller (raspberry pi). This is a very easy and
accurate method compared to using camera as a
recognising component. This is easily portable set up.
II. EXISTING

Keywords- Smart speaking system for mute people, Raspberry
Pi, Flex Sensors, Accelerometer Sensor.
I. INTRODUCTION
Sign-language has become a most common method of
communicating to those people who cannot speak. It is a
language that uses the hand motions to express alphabets
and words. People who are using the sign language were
recorded just in china alone . It exceedsupto 80 million in
total and especially those people will always have a
problem of communicating with each others who can’t
able to understand the sign language.
Vision method has become the popular method used
for sign recognition in the past decades. It is a system
which uses a camera to sense the information that has
been obtained through finger motions. It is the most
commonlyused visual-based method. It has been a
tremendous effort and has been gone into the
In the block diagram we will be having the component
like flex sensor, ADC converter ADC0804,
microcontroller such as PIC, Arduino and speaker. Here
the flex sensor used for detecting the hand motion. Flex
sensor is also called as bending sensor, based upon the
bending of sensor their resistance of the sensor get varied.
Then their voltage also varied, according to the voltage
value the command will be provided. So this flex sensor
is exactly act as a variable resistor. Flex sensor is analog
in nature. This input is directly given to Arduino. Because
an Arduino has inbuilt analog to digital converter. This
will convert the analog sensor input into digital input to
Arduino. But in the case of raspberry pi we have to use
ADC IC externally. Example ADC0804.

762
978-1-5386-9533-3/19/$31.00 ©2019 IEEE
 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS)

Fig. 1. Block diagram of existing system

Advantage of existing system is Reduction in

environmental effect, Portable Easy implementation,and
Small size. And the disadvantage is not accurate in all
direction Small variations are not determined.
Fig. 3. Image of flex sensor

III. PROPOSED SYSTEM A. Range: Its resistance will range between 30 - 40k.
B. Accuracy: They are not super accurate sensors. They
In this proposed system the dump people hand motion or
will give reasonable sense according to the amount of
gesture can be detected and then it will be converted into
bend.
human hearing voice signal.
C. Reasonability: It uses a carbon on a strip of plastic to
act like variable sensor.

Fig. 4. Basic circuit of Flex Sensor

V. ACCELEROMETER SENSOR

Fig. 2. Block diagram of proposed system

Then for two way communication the human voice will
also convert it into hand motion. And in addition to this
concept we will going to be add one more idea for their
safety purpose, which is if any emergency time their hand
motion information will automatically send to their
relation or friend’s mobile in the form of text.
Accelerometer Sensor is used to measure the proper
acceleration. Proper acceleration, being the acceleration
of a body in its own instantaneous rest frame, is not the
same as co-ordinate acceleration, being the acceleration
in the fixed co-ordinate system. It also measures the
tilting motion of the device

IV. FLEX SENSORS

Flex sensor is also known as bend sensor it can measure
the amount of deflection. The flex sensor will stuck to the
surface and its resistance is varied while bending the
surface. The resistance is directly proportional to the
amount of bend.

Fig. 5. Circuit of Accelerometer Sensor

763
 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS)

A. Range: It is the frequency range where the

accelerometer is at a specified deviation.
B. Accuracy: It is used to make accurate measurement.
C . Reasonability: It is used to measure acceleration
forces.

Fig. 8. Block diagram of module 2

Voice recognition software must be installed on the

raspberry pi 3. Then it will work with the help of the
internet. The library used here is node- red. Raspberry pi
3 has an inbuilt Wi-Fi and it will fit with the application
Fig. 6. Block diagram of accelerometer Sensor
very well. It is the main advantage of installing library
file, and most importantly internet may normally accessed
with an access point5 orit may be accessed via hotspot.
VI. SYSTEM DESIGN
In first module the flex sensor is used to detect the hand
motion or gesture and the accelerometer sensor (MEMS
sensor) is used to detect the three dimensional motion of a
hand. The controller used here is raspberry pi which does
not contain any analog to digital converter. Here both
sensors are analog in nature. So two ADC converter is
required for this system. One is for flex sensor and
another one is for accelerometer. ADC0804 is for flex
sensor and the MCP3008 is for accelerometer. These two
converter converts the analog signal into digital signal
and give to raspberry pi for further purpose.
Fig. 9. Basic diagram of IBM Bluemix

IBM BLUEMIX is used to perform voice recognition. In

Fig. 7. Block diagram of module 1
In second module normal people voice will be detected.
And convert the voice into hand gesture.
order to perform this operation internet connectivity is a
basic need to have for this project. In simpler way to say
that the conversion of speech to text conversion is easier.
The final result will be the text on giving input as a
speech using USB Microphone.
The items required for this module are
1. Raspberry pi
2. Internet access
Raspberry pi do not have an inbuilt sound card and
therefore it may not support microphones while using
audio jack.
In the third module, the mute people’s gesture is
automatically send by a text message to their close friends
or relatives. Here way to SMS app is used to send a
message from PI to mobile.

764
 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS)

Fig. 10. Block diagram of module 3

Fig. 11. Result

VII. CONCLUSION
In this paper we have designed a prototype
model for the dumb people. The important key factor of
this project is to facilitate these people from these risk
factor and to make them more confident to manage their
sites by themselves. The main advantages of this project
is to carry away this device very easily wherever the need
comes. It is of about less weight. Simple coding language
is used to make is less complicated. The project will be
beneficial to both dumb and as well as normal people in
this electronic world. We are trying to make this project
more and more convenient to mute people.

REFERENCE
[1] B.G.Lee, member IEEE and S.M.Lee “smart wearable hand
devices for sign language interpretation system with sensor fusion”,
volume 18 issue: 3, February 1, 2018 IEEE sensor Journal.
[2] Mohammed Elmahgiubi, “sign language translator and
gesture recognition”, 17 December 2015, IEEE.
[3] Lih-jen kau, member IEEE Bo-xun zhuo, “a real time
portable sign language translation system”, 26 January 2017, IEEE
Journal.
[4] Merin ary koshi, “a survey on advanced technology
communication between deaf/dumb people using eye blink and flex
sensor”, 01 February 2018, IEEE Journal.
[5] Safayet ahmed, “electronic speaking system for speech
impaired people: speak up”, 29 Oct 2015, IEEE Journal.

765