Authors:

Shaziya I. Shaikh, Diya R. Prajapati, Manali S. Brahmbhatt

Abstract:

Communication is a crucial aspect of human interaction and generally relies on a mutual language either verbally or non-verbally to construct that communication. However, in the context of society, there are many deaf and hard of hearing people who are not able to communicate using these common methods. This prevents them from gaining equal access to society by limiting their access to opportunities and ostracizing them. Here we propose a real-time system that recognizes US Sign Language – one of many sign languages from across the world – using a number of machine learning based image processing techniques.

This system makes use of Python using multiple external libraries and architectures- Media Pipe for real-time hand tracking, OpenCV for deep image processing, Tensorflow for state-of-the-art deep learning, and scikit-learn to refine the computer vision. A Random Forest Classifier is used to classify hand landmarks coming from MediaPipe for the static representation of the sign language.

In our system we build the foundational processing, feature extraction, and classification steps required to recognize a set of US Sign Language gestures with strong performance and generalization. The system can work in low-light and awkward lighting situations with occasional gesture misclassification and remain robust to occlusion, multiple gesture forms, and other encountered phenomena.

This has the potential to improve all aspects of life for persons who utilize sign language for communication as well as continue the progress in human-computer interaction and inclusive access for all humans. Future work will involve deep learning architectures, sample and preprocessing optimization, and increased dataset collection to create a classifier capable of recognizing complex sign language structures. This is just the beginning of future work at the intersection of sign language and computer vision.

Keywords:

Hand Gesture Recognition, American Sign Language, Machine Learning, Image Processing, Image Classification, TensorFlow, OpenCV, Scikit-Learn, Media pipe

Vol & Issue:

VOL.17, ISSUE No.4, December 2025