Abstract
Hand gestures serve as the primary means of communication in sign languages, which are composed of visual gestures made up of hands, faces, and other bodily motions. Although sign language has become more common in recent years, communicating with sign language speakers or signers remains difficult for non-sign language speakers. There has been promising progress in the disciplines of motion and gesture detection utilizing Artificial Intelligent techniques because of recent advances in deep learning and computer vision. The deep learning network makes full use of the advantages of time series classification provided by the recurrent neural network model as well as the feature extraction capabilities of convolutional neural networks to achieve more accurate recognition. High precision, scalability, and robustness, on the other hand, remain significant issues in sign language recognition research. The purpose of this research is to examine hybrid neural network to improve the accuracy and robustness of sign language recognition. This research proposes a sign language recognition system using an ensemble of convolutional neural network (CNN) models followed by a long short-term memory (LSTM) model. The proposed system is designed to recognize hand gestures and interpret sign language, with a focus on American Sign Language (ASL) and sign digit dataset. Research evaluated the performance of the proposed system using the same ASL dataset and achieved an accuracy of 99.3%. We compared the performance of the proposed model with standalone CNN models and found that the proposed hybrid model outperformed standalone models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Subburaj S, Murugavalli S (2022) Survey on sign language recognition in context of vision-based and deep learning. Meas Sens 23:100385. https://www.sciencedirect.com/science/article/pii/S2665917422000198
Wadhawan A, Kumar P (2021) Sign Language recognition systems: a decade systematic literature review. Arch Comput Methods Eng 283:785–813. https://doi.org/10.1007/s11831-019-09384-2
Rathi D (2018) Optimization of transfer learning for sign language recognition targeting mobile platform. CoRR, abs/1805.06618. http://arxiv.org/abs/1805.06618
Masood S, Thuwal HC, Srivastava A (2018) American sign language character recognition using convolution neural network. In: Satapathy Suresh Chandra DS, Bhateja (eds) Smart computing, and informatics. Springer Singapore, Singapore, pp 403–412
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. ICLR. http://arxiv.org/abs/1409.1556
Mannan A, Abbasi A, Javed AR, Ahsan A, Gadekallu TR, Xin Q (2022) Hypertuned deep convolutional neural network for sign language recognition. Comput Intell Neurosci 1450822. https://doi.org/10.1155/2022/1450822
Katoch S, Singh V, Tiwary US (2022) Indian sign language recognition system using SURF with SVM and CNN. Array 14:100141. https://www.sciencedirect.com/science/article/pii/S2590005622000121
Yirtici T, Yurtkan K (2022) Regional-CNN-based enhanced Turkish sign language recognition. SIViP 165:1305–1311. https://doi.org/10.1007/s11760-021-02082-2
Gupta N (2022) Sign language recognition using diverse deep learning models. In: Goutam G, Travieso-González CM et al (eds) International conference on artificial intelligence and sustainable engineering. Springer, Singapore, pp 463–475
Sharma S, Kumar K (2021) ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks. Multimedia Tools Appl 8017:26319–26331. https://doi.org/10.1007/s11042-021-10768-5
Bheda V, Radpour D (2017) Using deep convolutional networks for gesture recognition in American sign language. CoRR, abs/1710.06836. http://arxiv.org/abs/1710.06836
Rao GA, Syamala K, Kishore PV, Sastry ASCS (2018) Deep convolutional neural networks for sign language recognition. In: 2018 Conference on signal processing and communication engineering systems, SPACES, pp 194–197
Koller O, Zargaran S, Ney H, Bowden R (2018) Deep sign: enabling robust statistical continuous sign language recognition via hybrid CNN-HMMs. Int J Comput Vision 12612:1311–1325. https://doi.org/10.1007/s11263-018-1121-3
Kandel I, Castelli M, Popovič A (2021) Comparing stacking ensemble techniques to improve musculoskeletal fracture image classification. J Imaging 76. https://www.mdpi.com/2313-433X/7/6/100
Acknowledgements
The authors thank the UNITAR management for supporting the publication of this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Saxena, A., Sultanova, N., Mustafina, J., Ismail, N.L. (2024). Artificial Intelligence (AI) Empowered Sign Language Recognition Using Hybrid Neural Networks. In: Bee Wah, Y., Al-Jumeily OBE, D., Berry, M.W. (eds) Data Science and Emerging Technologies. DaSET 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 191. Springer, Singapore. https://doi.org/10.1007/978-981-97-0293-0_3
Download citation
DOI: https://doi.org/10.1007/978-981-97-0293-0_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0292-3
Online ISBN: 978-981-97-0293-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)