WHISPER TO WAVES:
CONVERTING SOUND
INTO SIGN LANGUAGE
USING PYTHON
Presented by:
K . Lalitha Abhigna
K . Uma Shanker
K . Abhishek
K . Venu
DOMAIN:
• PYTHON:
1) Python is commonly used in audio-to-sign language translators because of its versatility,
extensive libraries for machine learning, and ease of use, allowing us to implement robust
systems for speech recognition, natural language processing, and sign language generation.
2) Python's syntax is known for being clear and easy to read, which simplifies the development
and maintenance of complex systems.
3) Python has a rich ecosystem of libraries and frameworks specifically designed for audio
processing, speech recognition, and sign language generation, such as PyAudio, Speech
Recognition, and libraries for working with sign language datasets and models.
INTRODUCTION:
1) It is easy to find a wide number of sign languages all over the world and almost every spoken
language has its respective sign language, so there are about more than 200 languages
available.
2) American Sign Language (ASL) is well-known and the best studied sign language in the
world. The grammar of ASL has been applied to other sign languages especially as in British
Sign Language (BSL).
3) This section is not going to go further with details of a single sign language because each sign
language has its own rules.
4) The next section will aim to give a general description of the shared or common characteristics
between the different sign languages: origin, phonology, and syntax.
INTRODUCTION(CONTD
):
5) Text to sign language conversion is mainly focused on communication between ordinary people
and ordinary people and deaf-mute people.
6) Sign language paves the way for deaf-mute people to communicate .Sign language is a visual
language that is used by deaf and dumb as their mother tongue.
7) It is figure out about 240 sign language have exist for spoken language in the world. Sign
language is a type of language that uses hand movements, facial expressions and body language to
communicate.
8) It is used by the people who are deaf and people who can hear but cannot speak.
ABSTRACT:
The Whisper to wave is based on converting an audio signal receiver into text using a speech-to-text API.
Speech-to-text conversion comprises small, medium, and large vocabulary processing. Such systems
analyze and interpret vocal inputs, converting them into corresponding textual representations. This study
provides a comparative analysis of the technologies used in small, medium, and large vocabulary speech
recognition systems. The comparative assessment identifies the advantages and limitations of various
methodologies. The empirical evaluation demonstrates the impact of language models on enhancing the
accuracy of speech-to-text conversion systems. It involves processing speech data containing noisy
sentences and incomplete words. The findings indicate a significant improvement in accuracy for randomly
selected sentences compared to sequentially structured sentences. Additionally, this research focuses on
developing an effective communication interface for individuals with disabilities through the integration of
graphical hand gestures. The implementation leverages core principles of Natural Language Processing
(NLP) to bring the developed system into reality.
LITERATURE REVIEW:
S.NO Author et Paper Techniqu Merits Demerits IOT Techniqu Security
al(Year) Title e Device e Sensor Challeng
Used es
1. Tewari, Real Uses It It does Not Not Data
Soni Time natural effectivel not mentione mentione Privacy
Singh, Sign language y utilizes mention d d and
Turlapati, Languag processin CNN for evaluatio Protectio
Bhuva(20 e g to feature ns on n
21) Recogniti convert extractio large
on speech n datasets
Framewo into sign ensuring
rk language robustne
. ss.
2. An Khanvilka NLP and English- Limited Not Not Data
Advancement r, Akilan, Google to-ISL to Words mentione mentione Privacy &
in Speech to (2020). APIs. Translatio and d d User
Sign Language n. Letters. Confident
Translation iality.
using 3D
Avatar
Animator.
3. Increasing López Automati Compreh Limited Smart Not User
adaptability of Ludeña, c Speech ensive to Cameras. mentione Data
a speech into San Recogniti Speech- Spanish d Protectio
sign language Segundo, on (ASR) to-Sign Sign n&
translation Morcillo, using Translatio Language Privacy
system. López. HSMMs n. (SSL). Safeguar
(2018). and NLP. ds.
4. Hand Gesture Eirini CNN and Supports Limited Depth Not Data
Recognition Mathe, Discrete One- Generaliz Sensing mentione Privacy &
using a Alexandr Fourier Hand and ation cameras. d Unauthori
Convolution os Transfor Two-Hand Without zed
Neural Mitsou, m (DFT) Gestures. Large Access.
Network. Evaggelo for Image Datasets.
s Spyrou. Processin
g.
LIMITATIONS:
Paper title : An Advancement in Speech to Sign Language Translation
using 3D Avatar Animator.
-Limited to Word and Letter-Level Translation.
-Low Accuracy (77%).
-Dependence on Google APIs.
-Lack of Facial Expressions and Body Language.
-No Real-Time Interaction.
-Limited ISL Dataset for Training.
EXISTING SYSTEM:
• Speech-to-Text Systems- Convert spoken language into written text using APIs and machine
learning models.
limitation: Often require internet connectivity and large datasets.
• Voice Assistants (e.g., Google Assistant, Siri, Alexa): Use speech recognition for command
execution and information retrieval.
limitation : No visual output for speech like sign language.
• Closed Captioning Systems (e.g., YouTube Auto-Captions): Provide real-time subtitles for
videos and live content.
limitation: Limited to text output only.
PROPOSED SYSTEM:
• The system captures real-time speech through an audio input module.
• Speech is converted into text using a speech-to-text API.
• Natural Language Processing (NLP) is applied to analyze and refine the text.
• The processed text is mapped to corresponding sign language gestures.
• A graphical interface displays animated or visual sign language output.
• The system provides an easy-to-use and accessible user interface.
• It supports real-time conversion from speech to sign language.
• Pre-trained language models are used to enhance accuracy and contextual
understanding.
PROBLEM
STATEMENT:
• People with hearing impairments lack tools that convert spoken language
into sign language for real-time communication.
• Existing speech-to-text systems don’t work well in noisy environments and
don’t support sign language output.
• There is no integrated system that uses speech recognition and NLP to
translate voice into accurate, meaningful sign language gestures.
HARDWARE
REQUIREMENTS:
Processor Intel I5 or equivalent
RAM Minimum 8 GB
Storage 256 GB HDD/SSD
Graphic Card NVIDIA GT 1030
SOFTWARE
REQUIREMENTS:
Operating System Windows 10/11
Programming Language Python 3.8 or above
Libraries PyAudio , NLTK , OpenCV,
Matplotlib
Web browser Chrome or Firefox
SYSTEM DESIGN:
• SYSTEM ARCHITECTURE:
UML DIAGRAMS:
• USE CASE DIAGRAM:
SEQUENCE DIAGRAM:
CLASS DIAGRAM:
ACTIVITY DIAGRAM:
CODE:
THANK YOU
