A PROJECT REPORT ON

”Interpretation Of Indian Sign Language

Using Machine Learning”

SUBMITTED TO THE SAVITRIBAI PHULE PUNE UNIVERSITY, PUNE

IN THE PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE

AWARD OF THE DEGREE

BACHELOR OF ENGINEERING
(Information Technology)
BY
Student Name: Saloni Thete Exam No: B190298581
Student Name: Pooja Vinchu Exam No: B190298584
Student Name: Raj Nikam Exam No: B190298562
Student Name: Yuvraj Nemade Exam No: B190298547

DEPARTMENT OF INFORMATION
TECHNOLOGY
Maratha vidya Prasarak Samaj’s Karamveer Baburao
Ganpatrao Thakare College of Engineering
address
SAVITRIBAI PHULE PUNE UNIVERSITY
2021 -2022
 CERTIFICATE
This is to certify that the Project Entitled

”Interpretation Of Indian Sign Language Using Machine Learning”

Submitted by
Student Name: Saloni Thete Exam No: B190298581
Student Name: Pooja Vinchu Exam No: B190298584
Student Name: Raj Nikam Exam No:B190298562
Student Name: Yuvraj Nemade Exam No: B190298547
is a bonafide work carried out by them under the supervision of Ms. Rupali
Tajanpure and it is approved for the partial fulfillment of the requirement of
Savtribai Phule Pune university, Pune for the award of the degree of Bachelor
of Engineering (Information Technology).

Ms Rupali Tajanpure DR. V.R Sonawane

Guide H.O.D
Dept. of Information Technology. Dept. of Information Technolog

S.R Devane
Principal
KBTCOE

Place :- Nashik
Date:-
 Acknowledgments

Guidance is the key factor for achieving the goals. No work can be com-pleted
without correct help from experts, in this field. At this point I wishto express
my gratitude towards those who have made our project successful.

It is my great pleasure to acknowledge sense of gratitude to all, who made it

possible for me to complete this project with success. It gives me great plea- sure
to thank my project guide, Ms.Rupali Tajanpure for great support guidance and
encouragement from time to time during project work.

It is my sincere thanks to Dr.VR Sonawane for their help and guidance, Also
I would like to express my deep gratitude towards all the teaching staff for
their guidance during project presentations. I also extends my sincere thanks
to our beloved Principal S.R Devane for their support and guidance.

Student Name: Saloni Thete

Student Name: Pooja Vinvhu

Student Name: Raj Nikam

Student Name: Yuvraj Nemade

(B.E. Information Technology.)

 Abstract

Voice and Language is the main thing for human to communicate

with each other. Due to hearing ability we can understand thoughts
of each other. Even nowadays we can give commands using voice
recognition. But what if one absolutely cannot hear anything and
eventually cannot speak. So The Sign Language is the main com-
municating tool for hearing impaired and mute people, and also
to ensure an independent life for them, the automatic interpre-
tation of sign language is an extensive research area. With the
use of image processing and artificial intelligence, many techniques
and algorithms have been developed in this area. Every sign lan-
guage recognition system is trained for recognizing the signs and
converting them into required pattern. The proposed system aim
to provide speech to speechless, in this paper the double handed
Indian Sign Language is captured as a series of images and it’s pro-
cessed with the help of Python and then it’s converted to speech and
text.
Contents

1 Introduction 1
1.1 Overview ............................................................................................ 2
1.2 Motivation of the Project ................................................................... 2
1.3 Problem Definition and Objectives .................................................. 2
1.4 Objective ............................................................................................ 2
1.5 Project Scope Limitations ................................................................ 3
1.6 Limitations ........................................................................................ 3
1.7 Methodologies of Problem solving ................................................... 3

2 Literature Survey 5

3 Software requirement specification 7

3.1 Assumption and Dependencies .......................................................... 8
3.2 Functional Requirements .................................................................... 8
3.2.1 System Feature ..................................................................... 8
3.3 External interface requirements ......................................................... 8
3.3.1 user interface ......................................................................... 8
3.4 Hardware Interfaces ........................................................................... 8
3.5 Software Interfaces ............................................................................ 9
3.6 Nonfunctional Interfaces .................................................................. 10
3.6.1 Performance requirements................................................... 10
3.6.2 Safety requirements ............................................................. 10
3.6.3 Software Quality Attributes ................................................ 10
3.7 System requirements........................................................................ 11
3.7.1 Database requirements ........................................................ 11
3.8 SOFTWARE REQUIREMENTS: ............................................ 11
3.9 Hardware requirements ................................................................... 11
3.10 Analysis Module. SDLC Module to be applied .............................. 12
3.10.1 Overview of responsibilities of Developer ........................... 12

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
4 System Design 14
4.1 system Architecture.......................................................................... 15
4.2 Mathematical Model ........................................................................ 15
4.3 Data Flow Diagram.......................................................................... 16
4.4 UML DIAGRAMS ...................................................................... 18

5 Project Plan 22

6 Conclusion 24
6.1 conclusion ......................................................................................... 25
6.2 Future Work .................................................................................... 25
6.3 Applications ..................................................................................... 25

References 25

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
List of Figures

3.1 SDLC Model Diagram ...................................................................... 12

4.1 system Architecture.......................................................................... 15

4.2 Data Flow diagram .......................................................................... 17
4.3 Data Flow diagram .......................................................................... 17
4.4 Data Flow diagram .......................................................................... 18
4.5 Use case Diagram ............................................................................ 19
4.6 Activity Diagram .............................................................................. 20
4.7 Sequence Diagram ............................................................................ 21
 CHAPTER 1
INTRODUCTION
1.1 Overview
Sign languages are vivid on wide and world level. There are multiple sign
languages in world which are regular in use that are ASL (American Sign
Language ) ISL (Indian Sign Language), BSL ( Bangladesh Sign Language),
MSL (Malaysian Sign Language). These languages are Built and Developed
with lots of work and practical testing with intention of feasibility to the deaf
and dumb persons. Any language is created with its word and its meaning.
Sign Language is created as “Sign” and “Action of That Sign”. Because here
we are not able to make them understand meaning of sign by writing word.
As they are deaf and can not listen from birth so we can not teach them
words.

1.2 Motivation of the Project

We are motivated with aim to use new technologies for better humanity.We
found Machine learning like technologies can be used for conquering the
backwardness occurred because of this physical disability.

1.3 Problem Definition and Objectives

A random person if visited to deaf person and if deaf person is in problem
and trying to explain it then it is very difficult to understand what exactly
he is trying to say. Delay in detecting his Sign Language can turn into big
critical problem for that deaf person. These kind of people can not spend
normal life. They face communication issues at every point. Also they get
boundaries and limitations to their dreams and professional aims. Hence
they get demotivated and Inferiority Complex.

1.4 Objective
Objective is to give them ability to be expressive in ideas and thoughts. They can
get helped in increasing their motivation and confidence and it will help them to
think positively and to conquer that physical disability. To develop system with
using latest technologies and tools we are keeping objective to
overcome from this global level problem.

1.5 Project Scope Limitations

This system will definitely can become step into innovation of this global level
problem solution. Our system can be Prototype and Proof of Concept for
global level solution. This system can be used by Deaf and Deaf persons and
also normal person can have this system with them and deaf person can
perform sign in from of camera and sign can be converted to text or speech.

1.6 Limitations
Most people don’t understand sign language.
A child should not need to rely on a translator.
Fine motor issues lead to garbled signs.
I think that AAC is faster to learn than sign.

1.7 Methodologies of Problem solving

1. Requirement gathering and analysis:

In this step of waterfall we identify what are various requirements are need
for our project such are software and hardware required, database, and in-
terfaces.
2. System Design:

In this system design phase we design the system which is easily understood
for end user i.e. user friendly. We design some UML diagrams and data flow
diagram to understand the system flow and system module and sequence of
execution.
3. Implementation:
In implementation phase of our project we have implemented various mod-
ule required of successfully getting expected outcome at the different module
levels. With inputs from system design, the system is first developed in small
programs called units, which are integrated in the next phase. Each unit is
developed and tested for its functionality which is referred to as Unit Testing.
4. Testing:

The different test cases are performed to test whether the project module
are giving expected outcome in assumed time. All the units developed in the
implementation phase are integrated into a system after testing of each unit.
Post integration the entire system is tested for any faults and failures.
5. Deployment of System:

Once the functional and non-functional testing is done, the product is de-ployed
in the customer environment or released into the market.
6. Maintenance:

There are some issues which come up in the client environment. To fix those
issues patches are released. Also to enhance the product some better versions
are released. Maintenance is done to deliver these changes in the customer
environment. All these phases are cascaded to each other in which progress
is seen as flowing steadily downwards like a waterfall through the phases.
The next phase is started only after the defined set of goals are achieved for
previous phase and it is signed off, so the name ”Waterfall Model”. In this model
phases do not overlap.

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
 CHAPTER 2
LITERATURE SURVEY
 Literature Survey
Paper Name Author Name IEEE Description
A Novel Fea- A Novel Fea- 2018 In this paper, a
ture Extraction ture Extraction novel approach
for American for American to hand recog-
Sign Language Sign Language nition based on
Recognition Recognition edge of binary
image using2D
camera is
proposed.
An effective sign Sunmok Kim, 2018 we introduced
language learn- Yangho Ji, and a new sign lan-
ing with object Ki-Baek Lee guage learning
classification us- method that
ing ROI extracts hand
area as the
ROI before
learning, using
object detection
network
Dynamic Tool Prateek SG Ja- 2018 For classification
for American gadeesh J of ASL gestures,
Sign Language we have used
CNN classifier
and achieved a
cross validation
accuracy of
98.66
Classification 2019 Joint Conference Our work into
Strategies for on Digital Li- the crawling and
Generating braries (JCDL) classification of
Multi language sign language
corpus for sign videos sets the
language stage for more
human-centered
activities explor-
ing instantiation
and community
engagement.
Sign Lan- Sandrine Tor- 2020 We develop a
guage Recog- nay Marzieh, multilingual
nition Towards Razavi Mathew sign language
Multilingual Magimai.-Doss approach, where 2g1yh-
Dept of Inform Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N n2dashmikov2e0m
2Taeschhikno2l0o ,a2N 21e-n2t2
modeling is
also done with
target.
 CHAPTER 3
SOFTWARE REQUIREMENT
SPECIFICATION
3.1 Assumption and Dependencies
• User must require the Python .
• User has to install the Python on his pc.
• User has to login to the system.

3.2 Functional Requirements

3.2.1 System Feature
• To have understanding of the problem statement.

• To know what are the hardware and software requirements of proposed

system.

• To have understanding of proposed system.

• To do planning various activates with the help of planner.

• Designing, programming, testing etc.

3.3 External interface requirements

3.3.1 user interface
• friendly application to interact with user.

3.4 Hardware Interfaces

Since the application must run over the Internet, all the hardware shall re-
quire to connect Internet will be hardware interface for the system. As for
e.g. WAN LAN, Ethernet Cross-Cable.
3.5 Software Interfaces
Python: Python is an interpreted, high-level and general-purpose program-
ming language. Python’s design philosophy emphasizes code readability with its
notable use of significant indentation. Its language constructs and object-
oriented approach aim to help programmers write clear, logical code for small
and large-scale projects. Python is dynamically-typed and garbage-collected.
It supports multiple programming paradigms, including structured (partic-
ularly, procedural), object-oriented and functional programming. Python is
often described as a ”batteries included” language due to its comprehensive
standard library. Python was created in the late 1980s, and first released in
1991, by Guido van Rossum as a successor to the ABC programming
language. Python 2.0, released in 2000, introduced new features, such as list
comprehensions, and a garbage collection system with reference count-ing,
and was discontinued with version 2.7 in 2020.Python 3.0, released in 2008,
was a major revision of the language that is not completely backward-
compatible and much Python 2 code does not run unmodified on Python 3.
With Python 2’s end of-life (and pip having dropped support in 2021 ), only
Python 3.6.x and later are supported, with older versions still supporting e.g.
Windows 7 (and old installers not restricted to 64-bit Windows). Python in-
terpreters are supported for mainstream operating systems and available for
a few more (and in the past supported many more). A global community of
programmers develops and maintains CPython, a free and open-source refer-
ence implementation. A non-profit organization, the Python Software Foun-
dation, manages and directs resources for Python and CPython development. As
of January 2021, Python ranks third in TIOBE’s index of most popular
programming languages, behind C and Java, having previously gained sec-
ond place and their award for the most popularity gain for 2020.

Spyder: Spyder is an open-source cross-platform integrated devel-

opment environment (IDE) for scientific programming in the Python lan-
guage. Spyder integrates with a number of prominent packages in the sci-
entific Python stack, including NumPy, SciPy, Matplotlib, pandas, IPython,
SymPy and Cython, as well as other open-source software.It is released under
the MIT license. Initially created and developed by Pierre Raybaut in 2009,
since 2012 Spyder has been maintained and continuously improved by a team of
scientific Python developers and the community. Spyder is extensible with
first-party and third-party plugins, includes support for interactive tools for
data inspection and embeds Python-specific code quality assurance and in-
trospection instruments, such as Pyflakes, Pylint and Rope. It is available
cross-platform through Anaconda, on Windows, on macOS through Mac- Ports,
and on major Linux distributions such as Arch Linux, Debian, Fedora, Gentoo
Linux, openSUSE and Ubuntu. Spyder uses Qt for its GUI and is designed to use
either of the PyQt or PySide Python bindings.QtPy, a thin abstraction layer
developed by the Spyder project and later adopted by mul- tiple other packages,
provides the flexibility to use either backend.

DB SQLite: DB Browser for SQLite (DB4S) is a high quality, visual, open

source tool to create, design, and edit database files compatible with SQLite.
DB4S is for users and developers who want to create, search, and edit databases.
DB4S uses a familiar spreadsheet-like interface, and complicated SQL commands
do not have to be learned. Controls and wizards are available for users to: Create
and compact database files Create, define, modify and delete tables Create,
define, and delete indexes Browse, edit, add, and delete records Search records
Import and export records as text Import and export tables from/to CSV files
Import and export databases from/to SQL dump files Issue SQL queries and
inspect the results Examine a log of all SQL commands issued by the application
Plot simple graphs based on table orquery data

3.6 Nonfunctional Interfaces

3.6.1 Performance requirements
The performance of the functions and every module must be well. The overall
performance of the software will enable the users to work efficiently.
Performance of detect face should be fast. Performance of the providing virtual
environment should be fast.

3.6.2 Safety requirements

The application is designed in modules where errors can be detected. This
makes it easier to install and update new functionality if required.

3.6.3 Software Quality Attributes

Our software has many quality attribute that are given below:-
Adaptability: This software is adaptable by all users.
Availability: This software is freely available to all users. The availability
of the software is easy for everyone.

Maintainability: After the deployment of the project if any error occurs

then it can beeasily maintained by the software developer.

Reliability: The performance of the software is better which will increase

the reliability of the Software.

3.7 System requirements

3.7.1 Database requirements
• DB Browser sqlite

3.8 SOFTWARE REQUIREMENTS:

• Operating System:Windows10
• LDE:Pycharm,Spyder
• Programming Language:Python
• Database:DB Browser SQLite

3.9 Hardware requirements

• Hardware : intel core
• Speed : 2.80 GHz
• RAM : 8GB
• HardDisk : 500 GB
• Key Board: Standard Windows Keyboard
3.10 Analysis Module. SDLC Module to be
applied
Waterfall Model is a sequential model that divides software development
into different phases. Each phase is designed for performing specific activity
during SDLC phase. It was introduced in 1970 by Winston Royce. This is used
for our project. This model is simple and easy to understand and use. It is
easy to manage due to the rigidity of the model each phase has specific
deliverables and a review process. Waterfall model works well for smaller
projects where requirements are clearly defined and very well understood.

Figure 3.1: SDLC Model Diagram

3.10.1 Overview of responsibilities of Developer

• To have understanding of the problem statement.
• To know what are the hardware and software requirements of proposed
system.
 • To have understanding of proposed system.
• To do planning various activates with the help of planner.
• Designing, programming, testing etc.

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
 CHAPTER 4
SYSTEM DESIGN
4.1 system Architecture

Figure 4.1: system Architecture

4.2 Mathematical Model

Let S be the Whole system which consists:

S= IP, Pro, OP.

Where,

A. IP is the input of the system.

B. Pro is the procedure applied to the system to process the given input.

C. OP is the output of the system.

A. Input:
IP = u, F,.

Where,

1. u be the user.

2. F be set of files used for sending

B. Procedure:

B. Process

1. In this project capture the image from sign and compair with the dataset.

2. According to image stored in dataset voice alert message give to the user
.

C. Output:

• After sign detection the voice message alert to the user .

4.3 Data Flow Diagram

In Data Flow Diagram,we Show that flow of data in our system in DFD0 we
show that base DFD in which rectangle present input as well as output and
circle show our system,In DFD1 we show actual input and actual output of
system input of our system is text or image and output is rumor detected like
wise in DFD 2 we present operation of user as well as admin.
Figure 4.2: Data Flow diagram

Figure 4.3: Data Flow diagram

 Figure 4.4: Data Flow diagram

4.4 UML DIAGRAMS

Unified Modeling Language is a standard language for writing software blueprints.The
UML may be used to visualize,specify,construct and document the arti-
facts of a softwareintensive system.UML is process independent,although
optimally it should be used in process that is use case driven,architecture-
centric,iterative,and incremental.The Number of UML Diagram is available.

Use case Diagram.

Activity Diagram.

Sequence Diagram.
Figure 4.5: Use case Diagram
 Figure 4.6: Activity Diagram

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
Figure 4.7: Sequence Diagram
 CHAPTER 5
PROJECT PLAN
 Sr Week Duration Task
No.
1. week 1 4 Group
Formation
2. week 2 4 Topic
Selection
3. week 3 5 Search
Related
Information
4. week 4 5 Understand
Concept
5. week 5 5 Problem
Definition
6. week 6 12 Project
Planning
7. week 7 7 Literature
Survey
8. week 8 6 Modeling
design
9. week 9 2 Technical
Specification

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
CHAPTER 6
CONCLUSION
6.1 conclusion
Sign Language is a tool to reduce the communication gap between deaf-mute
people and normal person. This system which is proposed above gives the
methodology which aims to do the same as the two-way communication is
possible. This method proposed here facilitates the conversion on the sign
into speech. This overcomes the requirement of a translator since real time
conversion is used. The system acts a voice of the person who is deaf-mute. This
project is a step towards helping a specially challenged people. This can be
further enhanced by making it more user friendly, efficient, portable, compatible
for more signs and as well as dynamic signs. This can be further improvised so as
to making it compatible for the mobile phones using the built-in camera of the
phone. We can increase the distance at which it canbe used by using a longer
trans-receiver module or over Wi-Fi.

6.2 Future Work

In future work, proposed system can be developed and implemented using
Raspberry Pi. Image Processing part should be improved so that In future work,
proposed system can be developed and implemented using Raspberry Pi.
Image Processing part should be improved so that System would be ableto
communicate in both directions i.e.it should be capable of converting nor- mal
language to sign language and vice versa. We will try to recognize signs which
include motion. Moreover we will focus on converting the sequence of
gestures into text i.e. word and sentences and then converting it into the speech
which can be heard.

6.3 Applications
This app was designed by bilingual (English and ASL) Deaf people and is
meant to teach conversational ASL. Using more than 1000 videos, it’s packed
with features to make learning ASL fun and easy.

Dept of Inform
Daetpiot nofTIencfhonrm
oDloaegtpiyot,noNfTaIesnchfhioknrm
o2l0oa2gti1yo-,n2N
2Taeschhikno2l0o2g1y-,2N
2ashik 2021-22
REFERENCES
1 Neha Poddar, Shrushti Rao, Shruti Sawant, Vrushali Somavanshi, Prof.
Sumita Chandak (Feb 2015), “Study of Sign Language Translation us-
ing Gesture Recognition”, International Journal of Advanced Research
in Computer and Communication Engineering

2 Jayshree R. Pansare, Maya Ingle (2016),” Vision-Based Approach for

American Sign Language Recognition Using Edge Orientation Histogram”,
at 2016 International Conference on Image, Vision and Computing.

3 Arslan Arif, Syed Tahir Hussain Rizvi, Iqra Jawaid, Muhammad Adam
Waleed, Techno-Talk: An American Sign Language (ASL) Translator,
at CoDIT’16 - April 6-8, 2016, Malta

4 Justin K. Chen, Debabrata Sengupta, Rukmani Ravi Sundaram,” Sign

Language Gesture Recognition with Unsupervised Feature Learning”

5 Matheesha Fernando, Janaka Wijayanayaka,”Low cost approach for

Real Time Sign Language Recognition” at 2013 IEEE 8th International
Conference on Industrial and Information Systems, ICIIS 2013, Aug. 18-
20, 2013, Sri

6 Aarthi M Vijayalakshmi, “SIGN LANGUAGE TO SPEECH CON-

VERSION”, at 2016 FIFTH INTERNATIONAL CONFERENCE ON
RECENT TRENDS IN INFORMATION

7 Caixia Wu, Chong Pan,YufengJin, Shengli Sun, and Guangyi Shi Shaox-
ing, “Improvement of Chinese Sign Language Translation System based
on Collaboration of Arm and Finger Sensing Nodes”, At The 6th An-
nual IEEE International Conference on Cyber Technology in Automa-
tion, Control and Intelligent Systems June 19-22, 2016,

8 Poonam Chavan1, Prof. Tushar Ghorpade2, Prof. Puja Padiya, “In-

dian Sign Language to Forecast Text using Leap Motion Sensor and
RF Classifier” at 2016 Symposium on Colossal Data Analysis and Net-
working (CDAN)
 9 Ms.Manisha D. Raut, Ms. Pallavi Dhok, Mr. Ketan Machhale, Ms.
Jaspreet Manjeet Hora, “A System for Recognition of Indian Sign Lan-
guage for Deaf People using Otsu’s Algorithm”, International Research
Journal of Engineering and Technology (IRJET)

10 C. Swapna, Shabnam Shaikh “Literature survey on hand gesture recog-

nition for video processing” International Journal of Advanced Re-
search in Computer Science and Software Engineering (IJARCSSE