Abstract
Sign language is used to communicate a particular message over some universally known and accepted gestures. The speech and hearing challenged people use a specific combination of hand gestures and movements to convey a message. Despite the extensive research progress in Sign Language Detection, cost effective and performance effective solutions are still need of the day. Deep learning, and computer vision can be used to provide an effective solution to the user. This can be very helpful for the hearing and speech impaired people in seamless communication with others around as knowing sign language is not something that is common to all. In this work, a sign detector is developed, which detects various signs of the Sign Language used by speech impaired people. Here, data taken as input in the form of images is extensively used for both training and testing using machine learning. A custom Convolutional Neural Network (CNN) model to identify the sign from an image frame using Open-CV is developed and sentence construction of the detected signs is accomplished. A lot of images have been used as input for the purposes of training and testing. Many of the symbols in sign language could be rightly identified. A series of gestures are translated as text to the recipient.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bantupalli, K., Xie, Y.: American sign language recognition using deep learning and computer vision. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 4896–4899. IEEE (2018). https://doi.org/10.1109/BigData.2018.8622141
Cabrera, M.E., Bogado, J.M., Fermin, L., Acuna, R., Ralev, D.: Glove-based gesture recognition system. In: Adaptive Mobile Robotics, pp. 747–753 (2012). https://doi.org/10.1142/9789814415958_0095
He, S.: Research of a sign language translation system based on deep learning. In: 2019 International Conference on Artificial Intelligence and Advanced Manufacturing (AIAM), pp. 392–396. IEEE (2019). https://doi.org/10.1109/AIAM48774.2019.00083. International Conference on Trendz in Information Sciences and Computing (TISC 2012)
Herath, H.C.M., Kumari, W.A.L.V., Senevirathne, W.A.P.B., Dissanayake, M.B.: Image based sign language recognition system for Sinhala sign language. Sign 3(5), 2 (2013)
Geetha, M., Manjusha, U.C.: A vision based recognition of Indian sign language alphabets and numerals using b-spline approximation. Int. J. Comput. Sci. Eng. 4(3), 406–415 (2012)
Pigou, L., Dieleman, S., Kindermans, P.-J., Schrauwen, B.: Sign language recognition using convolutional neural networks. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 572–578. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_40
Escalera, S., et al.: ChaLearn looking at people challenge 2014: dataset and results. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 459–473. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_32
Huang, J., Zhou, W., Li, H.: Sign language recognition using 3D convolutional neural networks. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE, Turin (2015)
Jaoa Carriera, A.Z.: Quo Vadis, action recognition? A new model and the kinetics dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4724–4733. IEEE, Honolulu (2018)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 248–255. IEEE. Miami, FL, USA (2009)
Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: HMDB: a large video database for human motion recognition. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2556–2563. IEEE (2011)
Zhao, M., Bu, J., Chen, C.: Robust background subtraction in HSV color space. In: Proceedings of SPIE MSAV, vol. 1, p. 4861 (2002). https://doi.org/10.1117/12.456333
Chowdhury, A., Cho, S.J., Chong, U.P.: A background subtraction method using color information in the frame averaging process. In: Proceedings of 2011 6th International Forum on Strategic Technology, vol. 2, pp. 1275–1279. IEEE (2011). https://doi.org/10.1109/ifost.2011.6021252
Mehreen, H., Mohammad, E.: Sign language recognition system using convolutional neural network and computer vision. Int. J. Eng. Res. Technol. 09(12) (2020). deeplearningbooks.org: Convolutional Networks
https://learnopencv.com/. Accessed 03 Apr 2023
https://data-flair.training/blogs/sign-language-recognition-python-ml-opencv/. Accessed 03 Apr 2023
https://core.ac.uk/download/pdf/191337614.pdf. Accessed 03 Apr 2023
https://core.ac.uk/reader/191309220. Accessed 03 Apr 2023
https://www.acadpubl.eu/jsi/2017-117-20-22/articles/20/2.pdf. Accessed 03 Apr 2023
https://github.com/rrupeshh/Simple-Sign-Language-Detector. Accessed 03 Apr 2023
Shirbhate, R.S., Shinde, V.D., Metkari, S.A., Borkar, P.U., Khandge, M.A.: Sign language recognition using machine learning algorithm. Int. Res. J. Eng. Technol. 7(03), 2122–2125 (2020)
https://www.freecodecamp.org/news/weekend-projects-sign-language-and-static-gesture-recognition-using-scikit-learn-60813d600e79/. Accessed 03 Apr 2023
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Suguna Mallika, S., Sanjana, A., Vani Gayatri, A., Veena Naga Sai, S. (2023). Sign Language Interpretation Using Deep Learning. In: Morusupalli, R., Dandibhotla, T.S., Atluri, V.V., Windridge, D., Lingras, P., Komati, V.R. (eds) Multi-disciplinary Trends in Artificial Intelligence. MIWAI 2023. Lecture Notes in Computer Science(), vol 14078. Springer, Cham. https://doi.org/10.1007/978-3-031-36402-0_64
Download citation
DOI: https://doi.org/10.1007/978-3-031-36402-0_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36401-3
Online ISBN: 978-3-031-36402-0
eBook Packages: Computer ScienceComputer Science (R0)