SIGN LANGUAGE RECOGNITION SYSTEM

Team Members:
Ankit Raj (2001920130024)
Harshit Mishra (2001920130068)
Aditya Kumar (2001920130011)
Aarti (2001920130002)

Objective: Our solution will be very helpful for deaf and some physically challenged people in
communication with other people who don’t know how to understand American sign language (ASL). It will
also be able to classify the various hand gestures used for fingerspelling in American sign language (ASL).

Technology: Artificial Intelligence, Image Processing, Machine Learning, OpenCV, TensorFlow.

Problem Statement: Our project aims to not only overcome communication obstacles faced by the people
who are deaf and mute but also encourages a more inclusive society where communication is accessible and
understandable for everyone. Communication barriers persist for deaf and mute individuals due to the limited
understanding of sign languages like American Sign Language (ASL) among the wider community.

Proposed Solution: Traditional methods for bridging the gap between physically challenged and normal
people often lack efficiency and accessibility. A Sign Language Recognition System is a user-friendly, real-time
system that allows for immediate translation of American Sign Language (ASL) fingerspelling, reducing
communication barriers.

Methodology:

 Dataset Collection: Gather a comprehensive dataset of ASL gestures. Ensure diversity in hand shapes,
orientations, and backgrounds. Annotate each image in the dataset with the corresponding ASL label.
 Data Preprocessing: Resize all images to a consistent size to maintain uniformity. Normalize pixel
values to a common scale (e.g., 0 to 1) for better convergence during training. Augment the dataset with
techniques like rotation, flipping, and scaling to improve model generalization.
 Data Splitting: Divide the dataset into training, validation, and testing sets to assess model performance
accurately.
 Feature Extraction: Utilize OpenCV for extracting relevant features from the images, such as contours,
edges, or color histograms. Experiment with different feature extraction techniques to find the most
effective representation of ASL gestures.
 Model Selection: Choose a suitable deep learning architecture for image classification. Convolutional
Neural Networks (CNNs) are commonly used for image-related tasks. Employ a pre-trained model or
design a custom architecture based on the complexity of the ASL gestures.
 Model Training: Use TensorFlow as the deep learning framework to train the selected model. Feed the
preprocessed images into the model and optimize the model weights using a suitable optimizer.
 Validation and Evaluation: Evaluate the model on the validation set to monitor its performance during
training. Use the test set to assess the generalization capabilities of the model. Employ metrics such as
accuracy, precision, recall, and F1 score to evaluate the model's effectiveness.
 Integration with OpenCV: Integrate the trained model with OpenCV for real-time image capture and
prediction.
 Deployment: Deploy the Sign Language Recognition System to the desired platform, considering factors
like performance, accessibility, and user interface requirements.