SHRI VAISHNAV VIDYAPEETH VISHWAVIDYALAYA

Presentation – I

SIGN LANGUAGE TO
TEXT CONVERTOR
Team Members
• 19100BTCSEMA05472 - Aman Bind
• 19100BTCSEMA05469 - Aayush Ingole
• 19100BTCSEMA05484 - Gladwin Kurian
• 19100BTCSEMA05507 - Yash Goswami
CONTENT
1. Introduction
2. Literature Survey
 Proposed System
3. Problem Domain
4. Feasibility Study
 Economic Feasibility

 Technical Feasibility

 Operational Feasibility
5. References

27 August 2020 Sign Language to Text Convertor 2

INTRODUCTION
Verbal communication performed by humans is one of the most unique traits in the entire animal
kingdom. Humans have used communication as a tool to share and expand our knowledge of the
world. It is safe to say that humans have created settlements, societies, technologies, strategies and
more, only through efficient communication. In today’s world, communication between individuals
is essential to the development and maintenance of society. But unfortunately, some individuals with
hearing/speech disabilities are unable to perform this basic human interaction. This barrier in
communication alienates them from society and hinders effective communication. Since it is not
feasible to assume that every person who communicates with such disabled individuals knows sign
language, we need a method that will eradicate this communication barrier. In this project, we are
proposing one such method.

27 August 2022 Sign Language to Text Convertor 3

SIGN LANGUAGE IS A VISUAL LANGUAGE AND CONSISTS OF 3 MAJOR COMPONENTS

Fingerspelling Word level sign vocabulary Non-manual features

Used to spell words letter by Used for the majority of Facial expressions and tongue,
letter. communication. tongue mouth and body position.

In our project we primarily focus on producing a model which can recognize ”Word Level Sign
Vocabulary” based hand gestures in order to form a complete word by combining each gesture.
Some gestures we are aiming to train are as given in the above image.
20XX Pitch deck title 4
LITERATURE SURVEY
Purpose of this literature survey is to find out different approaches and
methods applied by various authors for translating sign language into
English language. Various models are studied for image processing and
image classification. Following table presents few researches of
different authors in this field:

27 August 2022 Sign Language to Text Convertor 5

STUDIES IN THIS FIELD
Title
American Sign Language
Recognition System: An
Optimal Approach.
Vision-based Portuguese Sign
Language Recognition
System.
A Comprehensive Analysis
on Sign Language
Recognition System
Conversion of sign language
into text
Sign language recognition
system using CNN and
Computer Vision
Research of a sign language
translation system based on
deep learning
A Survey on Sign Language
Recognition Systems
Author Publication Remark
Shivashankara S, International Journal of Image, Graphics ASL alphabets A-Z (26) and numbers 0- 9 (10) are recognized with 93.05% accuracy.
Srinath S and Signal Processing, Aug 2018 However, it is limited to letters and numbers.
Paulo Trigueiros, Springer Science and Business Media Sixteen models were compared based on precision, recall, accuracy, activation function,
Fernando R, Luís P R LLC, April 2014. and AROC. The variance in most models was analogous. Accuracy: 74%, Precision:
78% , Recall: 68%.
Rajesh George Rajan, International Journal of Recent Different Acquisition methods like SVM, HMM, SVM+ANN, PCA+KNN, MLP+MDC,
M Judith Leo Technology and Engineering (IJRTE), Polynomial classifier and ANFIS for sign language recognition
March 2019
Mahesh Kumar N B International Journal of Applied Recognizing Indian sign language with MATLAB, total of 10 images of each alphabet
Engineering Research (2018) has been taken. It used Linear Discriminant Analysis (LDA), because LDA reduces
dimensionality ultimately reducing noise, and achieving higher accuracy. It is limited to
Alphabets only.
Mehreen H, International Journal of Engineering CNN based model for classifying ASL with 10 alphabets having accuracy above 90%
Mohammad Research & Technology (IJERT), (with gloves), HSV colour algorithm for detecting gesture. The dataset is self created
EW December 2020 with 80:20 split. It is limited to 10 alphabets only.
Siming He Conference on artificial intelligence and Faster R-CNN with an embedded RPN module is used with accuracy of 91.7%, he used
advanced manufacturing ,2019 3D CNN for feature extraction and sign language recognition framework consisting of
LSTM.
Shruty M. Tomar, International Journal of Creative In This they Study various classification techniques concludes that deep neural network
Dr.Narendra M. research Thoughts (IJCRT) 2021 (CNN, Inception model, LSTM) performs better than traditional classifiers such as
Patel,Dr. Darshak G. T. KNN and SVM. LSTM had the best performance among all.
CONCLUSION
As the above table explains there are various methods/models to detect the sign
language. One thing which is frequently observed was the quantity and quality of
data set used to train the model. When the dataset size is increased the person
reaches more to the desired accuracy. Quality of data set affects the model
exponentially, where there is dataset made by web cam, data set of any higher
resolution, data augmentation, data set with continuous videos changes the
accuracy.

Out of various papers reviewed most of the papers are based on

CNN(Convolutional Neural Network) or CNN integrated models for image
classification. However, LSTM(Long Short-Term Memory) based model to
capture the important features and discard unwanted features.

27 August 2022 Sign Language to Text 7

DIFFERENCE BETWEEN CNN AND LSTM
CNN
• A convolutional neural network, or CNN, is a deep
learning neural network designed for processing
structured arrays of data such as images. CNN are
widely used in computer vision and have become the
state of the art for many visual applications such as
image classification and have also found success
in natural language processing for text classification.
27 August 2022
LSTM
Long Short-Term Memory is a kind of recurrent neural
network(RNN) which can retain the information for a
long period of time. It is used for processing,
predicting, and classifying based on time-series
data. Unlike CNN, LSTM is generally trained on
sequence of data i.e., sequence of images. It is much
faster than CNN for recognition of time-series data.
Sign Language to Text Convertor 8
PROBLEM
DOMAIN
• Limitations of Current Systems
• Proposed System

27 August 2022 Sign Language to Text Convertor 9

 LIMITATIONS OF CURRENT
SYSTEMS
• The persistent problem in the present Sign Language Recognition is that all the implemented systems
rely on static gesture recognition which is very slow or not handy to be used by speech impaired
people. In all the existing systems they use a single gesture for a single character which takes much
time and isn't efficient working of the system. 

• Most of the solutions available are either very expensive or business oriented, and not reachable
to general public.

27 August 2022 Sign Language to Text Convertor 10

PROPOSED SYSTEM
Aim of this project is to develop a concept of virtual talking system using a camera sensor for
people who are in need, this concept achieved by using image processing and human hand gesture
input. This mainly helps people who are deaf-mute.
• Implementing a single gesture for a word and single gesture for a phrase so that it will be more
natural.
• The problem statement revolves around the idea of a camera- based sign language recognition
system, so that the product will be Cost efficient.
• Objective of this project is to design a solution that is intuitive, simple and user friendly.
• Communication for normal society is not difficult. It should be the same way for the Speech
impaired individuals.

27 August 2022 Sign Language to Text Convertor 11

FEASIBILITY
STUDY
• Technical Feasibility
• Economical Feasibility
• Operational Feasibility

27 August 2022 Sign Language to Text Convertor 12

TECHNICAL
FEASIBILITY
This project is technically feasible in real world scenario. However it requires a
lot of computational power to recognise the hand gestures. Because of high
computational requirements, the system cannot be used on mobile devices . We
will use Google's TensorFlow to create the Machine Learning model for
recognising the hand movements or hand gestures on a fairly capable computing
hardware.

All of the technical software's are also freely available on the internet. We can
train the model on online notebooks provided by Kaggle for free, without
worrying about the hardware resources.

27 August 2022 Sign Language to Text Convertor 13

TECHNICAL REQUIREMENTS
SOFTWARE REQUIREMENTS HARDWARE REQUIREMENTS

• OpenCv • CPU: Core i5 10gen or Higher

• Tkinter/Kivy/PyQT • GPU: GeForce GTX 980 or higher
• TensorFlow • RAM: 8GB or Higher
• Media Pipe • ROM: 256GB or higher (SSD preferred)
• Jupiter Notebook
• Numpy, Matplotlib, Scikit Learn.
• OS: Windows 8 or higher/ MacOS 10
(Catalina) or higher/Ubuntu based OS
• IDE: VS Code/PyCharm/Sublime, etc.

20XX Pitch deck title 14

OPERATIONAL FEASIBILIT
Y
This project is feasible to operate on a Desktop or Laptop devices with modest
performance. However, this project is not operational on mobile devices like Android and
iPhone due to huge performance gap. Though it can be made operational on mobile
devices using TensorFlow lite, but accuracy will not be on par with the standard for
actual communication.

Thus, this project will be limited to desktop and laptop devices. We could have used web
browser-based approach for this project, but that would affect the economic feasibility of
this project. So, we will limit this project to desktop and laptops.

27 August 2022 Sign Language to Text Convertor 15

ECONOMIC
FEASIBILITY 
Development of this project is economically feasible, as it is made from fully
open-source technology. Thus, the cost of the project is very less . All the
components of this project are open source e.g., python for backend,
TensorFlow for training the model and OpenCV for recognition. All the
resources required to create the project are freely available and open source.

However, The data needed to train the model is not available, but data and
information required for constructing the model is freely available on the
internet.

27 August 2022 Sign Language to Text Convertor 16

REFERENCES
• Akshay Divkar, Rushikesh Bailkar, Dr. Chhaya S. Pawar, “Gesture Based Real-time Indian Sign Language Interpreter”, International Journal of Scientific Research
in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 7 Issue 3, pp. 387-394, May-June 2021. Available
at DOI : https://doi.org/10.32628/CSEIT217374 

• Hema B N., Sania Anjum, Umme Hani, Vanaja P., Akshatha M., ”Sign Language and Gesture Recognition for Deaf and Dumb People”, International Research
Journal of Engineering and Technology (IRJET) , e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072

• Ss, Shivashankara & S, Dr.Srinath. (2018). American Sign Language Recognition System: An Optimal Approach. International Journal of Image, Graphics and
Signal Processing. 10. 10.5815/ijigsp.2018.08.03. https://www.semanticscholar.org/paper/American-Sign-Language-Recognition-System%3A-An-Shivashankara-
Srinath/e01c778091d8f0ff54c02dcccd38ab40744832ae

• Shreyas Viswanathan, Saurabh Pandey, Kartik Sharma, Dr P Vijayakumar, “SIGN LANGUAGE TO TEXT AND SPEECH CONVERSION USING CNN”,
International Research Journal of Modernization in Engineering Technology and Science, e-ISSN: 2582-5208, Volume:03/Issue:05/May-2021, www.irjmets.com

• Shruty M. Tomar, Dr.Narendra M. Patel, Dr. Darshak G. T., “A Survey on Sign Language Recognition Systems”, International Journal of Creative research
Thoughts (IJCRT) 2021, Volume 9, Issue 3 March 2021 | ISSN: 2320-2882, https://ijcrt.org/IJCRT2103503.pdf

• Mahesh Kumar N B, “Conversion of sign language into text”, International Journal of Creative research Thoughts (IJCRT) 2021, ISSN 0973-4562 Volume 13,
Number 9 (2018) pp. 7154-7161

• He, Siming, “Research of a Sign Language Translation System Based on Deep Learning”, International Conference on Artificial Intelligence and Advanced
Manufacturing (AIAM), 2019, Publisher: IEEE, DOI: 10.1109/AIAM48774.2019.00083

27 August 2022 Sign Language to Text Convertor 17

THANK
YOU
 ?
QUESTION'S