Gesture Connect : Empowering Inclusive

Communication.

Project Guide Group Members

Dr Devi Devapal Akshay P R (CEM20CS008)
Asst . Professor Lubna Saghik S (CEM20CS030)
CEM Nandana Diji V (CEM20CS032)
Nasrin Fathima B T (CEM20CS034)
 WHAT IS GESTURE CONNECT?
 Gesture Connect is an AI-powered technology that translates sign language into speech .

 It was created to empower mute people by making communication more accessible and inclusive.

 It can be used in a variety of settings, including education, employment, and social settings

 It has the potential to revolutionize the way that mute people communicate with the world around them.

 OBJECTIVES
• Enable people to communicate effectively with each other regardless of their verbal language skills or abilities.

• Accessible to everyone.

• Create a more inclusive and equitable society, where everyone has a voice and feels valued.

• Reduce communication barriers and improve understanding between different cultures.

• Empower individuals to express themselves freely and creatively using gestures.

• Make the world a more connected and communicative place.


PAPER TITLE
Intelligent gloves: An IT intervention for deaf-
mute people [3].
A Novel Machine Learning Based Two-Way
Communication System for Deaf and Mute [2].
Review on Systems Based Sensory Gloves for
Sign Language Recognitions State of the Art
between 2007 and 2017 [5].
Gesture Based Communication Tool for Mute
and Hearing impaired People [1].
Gesture Recognition System For Translating
Indian Sign Language Into Text And Speech
[4].
LITERATURE SURVEY
ADVANTAGES
• It addresses a real-world problem of
communication between deaf-mutes and non-
deaf-mutes, which affects millions of people
around the world.
• It provides a low-cost and easy-to-use
solution that does not require custom-made
hardware or training for the users.
• The paper analyzes the device characteristics,
recognition methods, performance metrics,
and datasets of the SLR systems and identifies
the trends and gaps in this field.
• The paper proposes a system that can
translate hand gestures of mute and hearing
impaired people into spoken words using an
Arduino and a WI-FI module.
• The system offers a dependable and usable
communication tool that is portable and user-
friendly.
DISADVANTAGES
• It uses a wired system, which might limit the
user’s mobility and comfort. A wireless system
using Bluetooth or WiFi might be more flexible
and convenient.
• It may not be able to handle complex or
ambiguous sentences or expressions that
require context or inference.
• The paper does not address the ethical, social,
or cultural issues that may arise from the use
of SLR systems or the diversity and
complexity of sign languages.
• The paper does not mention the accuracy,
speed, or robustness of the system or
compare it with other existing systems.
• There could also be limitations related to the
cost of the hardware components, power
consumption, and the need for user training
to effectively use the system.
Glove Design
 METHOD IMPLEMENTATION

• Define Objectives and Features

• Research and Development

• Sensor Integration and Hardware Development

• Software Development

• Testing and Validation

• Manufacturing and Production

• Integration and Connectivity

 ACTION PLAN FOR IMPLEMENTATION

• Training data collection: Collect sign language collection for creation of training data.

• Pre-processing: Data is pre-processed for reading sensors from each sign language.

• Caliberation: Sensors are caliberated for most efficiency .

• Optimization :Neural model is created for maximum optimized output.

• Training: Realtime training is done for real time use.

GANTT CHART
 IMPLEMENTATION

 SOFTWARE REQUIREMENTS

• Text to speech api (Google Text to speech, Amazon Polly, IBM Watson Text to speech)

• Vs code

• Python
 HARDWARE REQUIREMENTS

• Flex Sensors

• It is typically a thin strip, making it lightweight and flexible.

• Designed to measure the amount of bending or deflection when subjected to physical flexing, the flex sensor provides essential

input for accurately capturing finger movements.

• As the sensor bends, it exhibits a change in electrical resistance, with the resistance value increasing as the sensor

is bent further.

• This unique characteristic allows the sensor to translate physical finger gestures into corresponding electrical signals.
• HC-05 Bluetooth device

• HC-05 is a class 2 Bluetooth module designed for transparent wireless serial communication.

• It is pre-configured as a slave Bluetooth device.

• Once it is paired to a master Bluetooth device such as PC, smart phones and tablet, its operation becomes

transparent to the user.

• TP4056 Battery

• The TP4056 is a complete constant-current/constant-voltage linear charger for single cell lithium-ion batteries.

• Its SOP package and low external component count make the them ideally suited for portable applications.

• It has current monitor, under voltage lockout, automatic recharge, and two status pin to indicate charge termination and the

presence of an input voltage.

• Accelerometer

• The MPU-6050 accelerometer is a device that measures proper acceleration.

• The MPU-6050 utilizes a thin, compact, and low-power design, making it suitable for measuring complete 3-axis acceleration.

It offers a base full-scale range of ±3g, providing precise readings.

• It features three axes (X, Y, and Z), and its sensing directions are orthogonal, meaning they have minimal cross-axis sensitivity.

• Output signals are provided in the form of analog voltages that are proportional to acceleration.

• Arduino

• The LilyPad Arduino is based on the ATmega328P microcontroller, which is part of the AVR family of microcontrollers.

• These boards have a circular or flower-like form factor, designed to be easily sewn into fabric or integrated into

wearable projects.

• It’s a control a connected device or system based on the recognized gestures. This could involve sending signals to another

electronic device, triggering LEDs or actuators, or interacting with software.

• The design includes:

i. Establish a Bluetooth connection.

ii. Compose and transmit sensor data.

iii. Receive incoming information.

iv. Sensor data to appropriate text through trained model.

v. Facilitate text-to-natural audio conversion

BLOCK DIAGRAM
FLOW CHART
 CONCLUSION

• Promise of Advancement

• Diverse Applications

• Seamless Interactions

• Transformative Impact

• Collaboration with the Deaf Community

• Ethical Considerations
 REFERENCES

[1]. Muhammad Imran Saleem , Atif Siddiqui , Shaheena Noor , Miguel-Angel LuqueNieto and Pablo Otero (2023)
A Gesture Based Communication Tool for Mute and Hearing impaired People.
[2].Muhammad Imran Saleem,Atif Siddiqui,Shaheena Noor,Miguel-Angel Luque-Nieto,
Pablo Otero(2022) A Novel machine learning based two-way communication system for deaf and mute.
[3]. Amal Babour, Hind Bitar, Ohoud Alzamzami, Dimah Alahmadi, Amal Barsheed, Amal Alghamdi, and
Hanadi Almshjary(2022) Intelligent gloves : An IT intervention for deaf mute people.
[4]. Jinsu Kunjumon and Dr. Rajesh Kannan Megalingam (2019) Hand Gesture Recognition System For
Translating Indian Sign Language Into Text And Speech.
[5]. Mohamed Aktham Ahmed, Bilal Bahaa Zaidan , Aws Alaa Zaidan, Mahmood Maher Salih and Muhammad
Modi bin Lakulu (2018) Review on SystemsBased Sensory Gloves for Sign Language Recognitions State of the
Art between 2007 and 2017.
THANK YOU.