article

American Sign Language word recognition with a sensory glove using artificial neural networks

Authors:

Ming C. LeuAuthors Info & Claims

Engineering Applications of Artificial Intelligence, Volume 24, Issue 7

Pages 1204 - 1213

https://doi.org/10.1016/j.engappai.2011.06.015

Published: 01 October 2011 Publication History

Abstract

An American Sign Language (ASL) recognition system is being developed using artificial neural networks (ANNs) to translate ASL words into English. The system uses a sensory glove called the Cyberglove(TM) and a Flock of Birds^(R) 3-D motion tracker to extract the gesture features. The data regarding finger joint angles obtained from strain gauges in the sensory glove define the hand shape, while the data from the tracker describe the trajectory of hand movements. The data from these devices are processed by a velocity network with noise reduction and feature extraction and by a word recognition network. Some global and local features are extracted for each ASL word. A neural network is used as a classifier of this feature vector. Our goal is to continuously recognize ASL signs using these devices in real time. We trained and tested the ANN model for 50 ASL words with a different number of samples for every word. The test results show that our feature vector extraction method and neural networks can be used successfully for isolated word recognition. This system is flexible and open for future extension.

References

[1]

Abulafya, N., 1995. Neural Networks for System Identification and Control, Ms. Thesis, University of London.

[2]

Allen, M.J., Asselin, P.K., Foulds, R., 2003. American Sign Language finger spelling recognition system. In: Proceedings of the 29th Bioengineering Conference Proceeding, IEEE, pp. 22-23.

[3]

. Linstok Pr, Silver Spring, MD.

[4]

Image processing system for interpreting motion in American Sign Language. J. Biomed. Eng. v14. 419-425.

[5]

Grobel, K., Assan, M., 1996. Isolated sign language recognition using hidden Markov Models. In: Proceedings of the International Conference on System, Man and Cybernetics, pp. 162-167.

[6]

Kramer, J., Leifer, L.J., 1990. A 'Talking Glove' for Nonverbal Deaf Individual. Technical Report CDR TR 1990 0312, Center for Design Research, Stanford University.

[7]

Kramer, J., 1996. The Talking-Glove: Hand-Gesture-to-Speech Using an Instrumented Glove and a Tree-Structured Neural Classifying Vector Quantizer, Ph.D. Thesis, Stanford University.

[8]

Lee, C., Xu, Y., 1996. Online, interactive learning of gestures for human/robot interface. Proceedings of the IEEE International Conference on Robotics and automation, Minneapolis, MN, vol. 4, pp. 2982-2987.

[9]

Recognition of hand gesture to human-computer interaction. IEEE. 2122-2177.

[10]

An introduction to computing with neural nets. IEEE ASSP. 2-22.

[11]

Murakami, K., Taguchi, H., 1991. Gesture recognition using recurrent neural networks. In: CHI'91 Conference Proceedings, Human Interface Laboratory, Fujitsu Laboratories, ACM, pp. 237-242.

[12]

Adaptive control using neural networks. In: Miller, W.T., Sutton, R.S., Werbos, P.J. (Eds.), Neural Networks for Control. Third Printing, MIT.

[13]

Oz, C., Sarawate N.N., Leu, M.C., 2004. American Sign Language Word Recognition with a Sensory Glove Using Artificial Neural Networks, ASME Press, Intelligent Engineering Systems through Artificial Neural Networks, vol. 14, ISBN 0-7918-0228-0, pp. 633-638.

[14]

Linguistic properties based on American Sign Language recognition with artificial neural network using a sensory glove and motion tracker. Neurocomputing. v70. 2891-2901.

[15]

Vehicle License Plate recognition with Artificial Neural Network. Springer.

[16]

Oz, C., Ercal, F., 2003b. Off-line Signature Recognition with ANN using Moment Invariant Method, Asme-press, Intelligent Engineering Systems through Artificial Neural Networks, vol. 14, ISBN:0-7918-0228-0, pp. 633-638.

[17]

A tutorial on hidden Markov models and selected applications in speech recognition. Proceeding of IEEE. v77 i2. 257-286.

[18]

Hand gesture recognition using a neural neural network shape fitting technique. Engineering Applications of Artificial Intelligence. v22. 1141-1158.

[19]

American Sign Language Dictionary (Revised Edition). Harper Perennial.

[20]

Stokoe, W.C., 1960. Sign language structure: an outline of the visual communication system of the American deaf. Studies Linguistics: Occasional Papers (No. 8). Buffalo: Dept. of Anthropology and Linguistics, University of Buffalo.

[21]

Dictionary of American Sign Language on Linguistic Principles. Linstok Press.

[22]

A survey of glove-based input. IEEE Computer and Applications. 30-39.

[23]

Gesture coding based in experiments with a hand gesture interface device. SIGCHI Bulletin. v23 i2. 67-73.

[24]

HMM-separation-based speech recognition for a distant moving speaker. IEEE Transactions on Speech and Audio Processing. v9 i2. 127-140.

[25]

Vogler, C., Metaxas, D., 1997. Adapting hidden Markov models for ASL recognition using three-dimensional computer vision methods. In: Proceeding of the IEEE International Conference on Systems, Man and Cybernetics, Orlando, pp. 12-15.

[26]

Vogler, C., Metaxas, D., 1999. Parallel hidden Markov models for American Sign Language recognition. In: IEEE proceeding of International Computer Vision Conference, September, pp. 116-122.

[27]

Isolated ASL recognition system for deaf persons. IEEE Transactions on Rehabilitation Engineering. v3 i3. 261-271.

[28]

Wang, H.G., Sarawate, N.N., Leu, M.C., 2004. Recognition of American Sign Language Gestures with a Sensory Glove, Japan-USA Symposium on Flexible Automation, Denver, CO.

[29]

American Sign Language Linguistic and Applied Dimensions. second ed. College-Hill Press.

[30]

Wysoski, S.G., Lamar, M.V., Kuroyanagi, S., Iwata, A., 2002. A rotation invariant approach on static gesture recognition using boundary histograms and neural networks, In: Proceeding of the Ninth International Conference on Neural Information (ICONIP'02), vol. 4, pp. 2137-2141.

Cited By

Parthasarathy NPalanichamy Y(2023)Novel Video Benchmark Dataset Generation and Real-Time Recognition of Symbolic Hand Gestures in Indian Dance Applying Deep Learning TechniquesJournal on Computing and Cultural Heritage 10.1145/358620716:3(1-19)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3586207
Bahia NRani R(2023)Multi-level Taxonomy Review for Sign Language Recognition: Emphasis on Indian Sign LanguageACM Transactions on Asian and Low-Resource Language Information Processing10.1145/353025922:1(1-39)Online publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1145/3530259
Qahtan SAlsattar HZaidan ADeveci MPamucar DMartinez L(2023)A comparative study of evaluating and benchmarking sign language recognition system-based wearable sensory devices using a single fuzzy setKnowledge-Based Systems10.1016/j.knosys.2023.110519269:COnline publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110519
Show More Cited By

Index Terms

American Sign Language word recognition with a sensory glove using artificial neural networks
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Hardware
  1. Power and energy
    1. Power estimation and optimization
      1. Platform power issues

Index terms have been assigned to the content through auto-classification.

Recommendations

Linguistic properties based on American Sign Language isolated word recognition with artificial neural networks using a sensory glove and motion tracker

Sign language (SL), which is a highly visual-spatial, linguistically complete, and natural language, is the main mode of communication among deaf people. Described in this paper are two different American Sign Language (ASL) word recognition systems ...
Linguistic properties based on american sign language recognition with artificial neural networks using a sensory glove and motion tracker
IWANN'05: Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems

Sign language, which is a highly visual-spatial, linguistically complete and natural language, is the main mode of communication among deaf people. In this paper, an American Sign Language (ASL) word recognition system is being developed using ...
American Sign Language Recognition for Alphabets Using MediaPipe and LSTM
Abstract
With the advancement of today's technologies in artificial intelligence, humans tend to use hand gestures in their communication to convey their ideas. Gesture recognition is an active area of research in the human-computer interface (HCI). ...

Comments

Information & Contributors

Information

Published In

cover image Engineering Applications of Artificial Intelligence

Engineering Applications of Artificial Intelligence Volume 24, Issue 7

October, 2011

208 pages

ISSN:0952-1976

Issue’s Table of Contents

Copyright © Elsevier Ltd © 2011.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 October 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Parthasarathy NPalanichamy Y(2023)Novel Video Benchmark Dataset Generation and Real-Time Recognition of Symbolic Hand Gestures in Indian Dance Applying Deep Learning TechniquesJournal on Computing and Cultural Heritage 10.1145/358620716:3(1-19)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3586207
Bahia NRani R(2023)Multi-level Taxonomy Review for Sign Language Recognition: Emphasis on Indian Sign LanguageACM Transactions on Asian and Low-Resource Language Information Processing10.1145/353025922:1(1-39)Online publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1145/3530259
Qahtan SAlsattar HZaidan ADeveci MPamucar DMartinez L(2023)A comparative study of evaluating and benchmarking sign language recognition system-based wearable sensory devices using a single fuzzy setKnowledge-Based Systems10.1016/j.knosys.2023.110519269:COnline publication date: 7-Jun-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110519
Katılmış ZKarakuzu C(2023)Double handed dynamic Turkish Sign Language recognition using Leap Motion with meta learning approachExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120453228:COnline publication date: 15-Oct-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120453
Huang YHuang JWu XJia Y(2022)Dynamic Sign Language Recognition Based on CBAM with Autoencoder Time Series Neural NetworkMobile Information Systems10.1155/2022/32477812022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/3247781
Mannan AAbbasi AJaved AAhsan AGadekallu TXin Q(2022)Hypertuned Deep Convolutional Neural Network for Sign Language RecognitionComputational Intelligence and Neuroscience10.1155/2022/14508222022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/1450822
Singh SChaturvedi A(2022)A reliable and efficient machine learning pipeline for american sign language gesture recognition using EMG sensorsMultimedia Tools and Applications10.1007/s11042-022-14117-y82:15(23833-23871)Online publication date: 17-Nov-2022
https://dl.acm.org/doi/10.1007/s11042-022-14117-y
Shah SKhan JAbbas SGhani A(2022)Symmetric mean binary pattern-based Pakistan sign language recognition using multiclass support vector machinesNeural Computing and Applications10.1007/s00521-022-07804-235:1(949-972)Online publication date: 24-Sep-2022
https://dl.acm.org/doi/10.1007/s00521-022-07804-2
Bautista Garcia MFeria Revano Jr. TCunanan Yabut A(2021)Hand Alphabet Recognition for Dactylology Conversion to English Print Using Streaming Video SegmentationProceedings of the 9th International Conference on Computer and Communications Management10.1145/3479162.3479169(46-51)Online publication date: 16-Jul-2021
https://dl.acm.org/doi/10.1145/3479162.3479169
Kruthiventi S S SJose GTandon NBiswal RKumar AShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Fingerspelling Recognition in the Wild with Fixed-Query based Visual AttentionProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475580(4362-4370)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475580
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents