Major Front Pagedocx

Real Time Sign Language Recognition
A Major Project Report
Submitted by
Ankit Sinha Ray [RA2011029010053]
Amit Raj [RA2011029010060]
Under the Guidance of
Dr. Dayana D S
Assistant Professor,
Department of Networking and Communications
In partial fulfilment of the requirements for the degree of

BACHELOR OF TECHNOLOGY
in
COMPUTER SCIENCE AND ENGINEERING
with a specialization in Computer Networking
DEPARTMENT OF NETWORKING AND COMMUNICATIONS

SCHOOL OF COMPUTING
COLLEGE OF ENGINEERING AND TECHNOLOGY
SRM INSTITUTE OF SCIENCE AND TECHNOLOGY
(Under Section 3 of UGC Act, 1956)
SRM NAGAR, KATTANKULATHUR – 603 203
CHENGALPATTU DISTRICT
April 2024
ii
SRM Institute of Science and Technology
Own Work Declaration Form
Degree/Course : B.Tech in Computer Science and Engineering
Student Name : Ankit Sinha Ray, Amit Raj
Registration Number: RA2011029010053, RA2011029010060.
Title of Work : Real Time Sign Language Recognition
We hereby certify that this assessment compiles with the University’s Rules and Regulations
relating to Academic misconduct and plagiarism, as listed in the University Website,
Regulations, and the Education Committee guidelines.
We confirm that all the work contained in this assessment is our own except where indicated,
and that we have met the following conditions:
▪ Clearly references / listed all sources as appropriate
▪ Referenced and put in inverted commas all quoted text (from books, web,etc.)
▪ Given the sources of all pictures, data etc that are not my own.
▪ Not made any use of the report(s) or essay(s) of any other student(s)either past or present
▪ Acknowledged in appropriate places any help that I have received from others (e.g fellow
students, technicians, statisticians, external sources)
▪ Compiled with any other plagiarism criteria specified in the Course hand book / University
website
I understand that any false claim for this work will be penalized in accordance with the
University policies and regulations.
DECLARATION:
I am aware of and understand the University’s policy on Academic misconduct and plagiarism
and I certify that this assessment is my / our own work, except where indicated by referring, and
that I have followed the good academic practices noted above.
Student 1 Signature:
Student 2 Signature:
Date:
If you are working in a group, please write your registration numbers and sign with the date
for every student in your group.
SRM INSTITUTE OF SCIENCE AND TECHNOLOGY
KATTANKULATHUR–603 203
BONAFIDE CERTIFICATE
Certified that 18CSP109L / I8CSP111L project report titled “Real Time Sign Language
Recognition” is the bonafide work of Ankit Sinha Ray [RegNo:RA2011029010053] and
Amit Raj [RegNo:RA2011029010060] who carried out the project work under my
supervision. Certified further, that to the best of my knowledge the work reported here does
not form part of any other thesis or dissertation on the basis of which a degree or award was
conferred on an earlier occasion for this or any other candidate.
Dr. Dayana DS Dr. Visalakshi P

Dr. ANNAPURANI K
SUPERVISOR PROFESSOR AND HOD

PANEL HEAD
Assistant Professor Department Assistant Professor

Assistant Professor
of Networking And Department of Networking And
Department of Networking and Communications Communications
Communications
ACKNOWLEDGEMENT
We express our humble gratitude to Dr. C. Muthamizhchelvan, Vice-Chancellor, SRM

Institute of Science and Technology, for the facilities extended for the project work and his
continued support.
We extend our sincere thanks to Dean-CET, SRM Institute of Science and Technology,
Dr.T.V. Gopal, for his invaluable support.
We wish to thank Dr. Revathi Venkataraman, Professor & Chairperson, School of
Computing, SRM Institute of Science and Technology, for her support throughout the project
work.
We are incredibly grateful to our Head of the Department, Dr. Annapurani K, Professor and
Head, Department of Networking and Communications, School of Computing, SRM Institute
of Science and Technology, for her suggestions and encouragement at all the stages of the
project work.
We want to convey our thanks to our Project Panel Head, Dr. Visalakshi P. Department of
Networking and Communications, School of Computing, SRM Institute of Science and
Technology, for their inputs during the project reviews and support.
We register our immeasurable thanks to our Faculty Advisor, Dr. T. Balachander,
Department of Networking and Communications, School of Computing, SRM Institute of
Science and Technology, for leading and helping us to complete our course.
Our inexpressible respect and thanks to our guide, Dr. Dayana DS., Assistant Professor,
Department of Networking and Communications, SRM Institute of Science and Technology,
for providing us with an opportunity to pursue our project under his mentorship. He provided
us with the freedom and support to explore the research topics of our interest. His passion for
solving problems and making a difference in the world has always been inspiring.
We sincerely thank the Networking and Communications, Department staff and students,
SRM Institute of Science and Technology, for their help during our project. Finally, we
would like to thank parents, family members, and friends for their unconditional love,
constant support, and encouragement.
Amit Raj [RA2011029010060]

Ankit Sinha Ray [RA2011029010053]
TABLE OF CONTENT
Sr. no Title Page No.
Abstract 1
List of figures 2
List of Symbols and Abbreviations 3
1 Introduction 4
1.1 General
1.2 Importance of sign language recognition
1.3 Introduction about the System
2 Literature Review 6
3 System Analysis 8
3.1 Approach and Methodology
3.1.1 Classifier Development
3.1.2 Dataset and features
3.1.3 Experiments Results and Analysis
4 Result 17
4.1 System architecture
4.2 Use case Diagram
4.3 Loss and Accuracy
4.4 Confusion Matrix
4.5 Real time user testing
4.6 Translation module
5 Challenges and Future Scope 24

6 References 27
7 Appendix 28
ABSTRACT
The invention of a fingerspelling translator that can translate Indian Sign Language (ISL) in
real time is a big step forward in the process of facilitating communication between the deaf
community and the general public. Within the scope of this research, we provide a novel
method that is founded on a convolutional neural network (CNN) for the purpose of
facilitating the proper translation of sign language fingerspelling gestures. We make use of a
pre-trained GoogLeNet architecture, which was first trained on the ILSVRC2012 dataset, and
we modify it so that it can do the unique job of ISL fingerspelling recognition. This is
accomplished through the utilisation of transfer learning. The ISL datasets from Surrey
University and Massey University are used to train our model, which has shown findings that
are consistent and promising. Specifically, our robust model is able to accomplish accurate
classification of letters a-e with first-time users, but another model is able to effectively
classify letters a-k in the majority of instances. Our findings are encouraging, indicating that
there is the potential for additional study and data gathering to have the effect of producing a
translator that is fully generalizable. This is despite the fact that the present datasets have
limitations. The purpose of this effort is to create the groundwork for future endeavours that
will be focused at improving the accessibility and inclusivity of communication for those
who use International Sign Language (ISL), with the ultimate objective of developing an ISL
translator that is both comprehensive and broadly applicable.
LIST OF FIGURES
Table 1. Optimal accuracy ranges for all models trained on each letter subset.
Fig. 2: Epochs vs. validation accuracy for all models trained on letters a-y (excluding j).
Fig. 3: Epochs vs. training loss for all models trained on letters ay (excluding j)
Fig. 4: Epochs vs. validation accuracy for the 2_init models trained on each letter subset
(excluding j)
Fig. 5: Epochs vs. training loss for the 2_init models trained on each letter subset (excluding
j)
Fig. 6: Confusion matrix for the 2_init model trained on letters ay (excluding j)
Fig. 7: Confusion matrix for the 2_init model trained on letters ak (excluding j)
LIST OF SYMBOLS AND ABBREVIATIONS
1. DB – Data Base
2. SMC - Secure Multiparty Computation
3. ISL - Indian sign language
4. ASL - American sign language
5. OpenCV - Open Computer Vision

Major Front Pagedocx

Uploaded by

Copyright:

Available Formats

Major Front Pagedocx

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Major Front Pagedocx

Uploaded by

Copyright:

Available Formats

Real Time Sign Language Recognition

A Major Project Report

In partial fulfilment of the requirements for the degree of

DEPARTMENT OF NETWORKING AND COMMUNICATIONS

Recognition” is the bonafide work of Ankit Sinha Ray [RegNo:RA2011029010053] and

conferred on an earlier occasion for this or any other candidate.

Dr. Dayana DS Dr. Visalakshi P

SUPERVISOR PROFESSOR AND HOD

Assistant Professor Department Assistant Professor

We express our humble gratitude to Dr. C. Muthamizhchelvan, Vice-Chancellor, SRM

Amit Raj [RA2011029010060]

5 Challenges and Future Scope 24

You might also like