research-article

Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features

Authors:

Pat Jangyodsuk,

Christopher Conly,

Vassilis AthitsosAuthors Info & Claims

PETRA '14: Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments

Article No.: 50, Pages 1 - 6

https://doi.org/10.1145/2674396.2674421

Published: 27 May 2014 Publication History

Abstract

Recognizing sign language is a very challenging task in computer vision. One of the more popular approaches, Dynamic Time Warping (DTW), utilizes hand trajectory information to compare a query sign with those in a database of examples. In this work, we conducted an American Sign Language (ASL) recognition experiment on Kinect sign data using DTW for sign trajectory similarity and Histogram of Oriented Gradient (HoG) [5] for hand shape representation. Our results show an improvement over the original work of [14], achieving an 82% accuracy in ranking signs in the 10 matches. In addition to our method that improves sign recognition accuracy, we propose a simple RGB-D alignment tool that can help roughly approximate alignment parameters between the color (RGB) and depth frames.

References

[1]

J. Alon, V. Athitsos, Q. Yuan, and S. Sclaroff. A unified framework for gesture recognition and spatiotemporal gesture segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(9):1685--1699, 2009.

Digital Library

[2]

F.-S. Chen, C.-M. Fu, and C.-L. Huang. Hand gesture recognition using a real-time tracking method and hidden markov models. Image and Vision Computing, 21(8):745--758, 2003.

[3]

C. Conly, P. Doliotis, P. Jangyodsuk, R. Alonzo, and V. Athitsos. Toward a 3d body part detection video dataset and hand tracking benchmark. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '13, pages 2:1--2:6, New York, NY, USA, 2013. ACM.

Digital Library

[4]

A. Corradini. Dynamic time warping for off-line recognition of a small gesture vocabulary. In Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001. Proceedings. IEEE ICCV Workshop on, pages 82--89. IEEE, 2001.

Digital Library

[5]

N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 886--893. IEEE, 2005.

Digital Library

[6]

S. R. Eddy. Hidden markov models. Current opinion in structural biology, 6(3):361--365, 1996.

[7]

A. Kar. Skeletal tracking using microsoft kinect. Methodology, 1: 1--11, 2010.

[8]

L. Karlinsky, M. Dinerstein, D. Harari, and S. Ullman. The chains model for detecting parts by their context. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 25--32, 2010.

[9]

J. Lafferty, A. McCallum, and F. C. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001.

Digital Library

[10]

A. Mittal, A. Zisserman, and P. H. S. Torr. Hand detection using multiple proposals. In British Machine Vision Conference, 2011.

[11]

T. Starner and A. Pentland. Real-time american sign language recognition from video using hidden markov models. In Motion-Based Recognition, pages 227--243. Springer, 1997.

[12]

G. Ten Holt, M. Reinders, and E. Hendriks. Multi-dimensional dynamic time warping for gesture recognition. In Thirteenth annual conference of the Advanced School for Computing and Imaging, volume 300, 2007.

[13]

e. a. Vondrick Carl. Hoggles: Visualizing object detection features, Jan. 2013.

[14]

H. Wang, R. Stefan, S. Moradi, V. Athitsos, C. Neidle, and F. Kamangar. A system for large vocabulary sign search.

[15]

S. B. Wang, A. Quattoni, L.-P. Morency, D. Demirdjian, and T. Darrell. Hidden conditional random fields for gesture recognition. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 2, pages 1521--1527. IEEE, 2006.

Digital Library

[16]

A. D. Wilson and A. F. Bobick. Parametric hidden markov models for gesture recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 21(9):884--900, 1999.

Digital Library

[17]

R. Yang and S. Sarkar. Gesture recognition using hidden markov models from fragmented observations. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 766--773. IEEE, 2006.

Digital Library

[18]

V. M. Z. Zhang, W. H. Liu and V. Athitsos. A viewpoint-independent statistical method for fall detection. In International Conference on Pattern Recognition, pages 3626--3630, Nov 2012.

[19]

Z. Zhang, R. Alonzo, and V. Athitsos. Experiments with computer vision methods for hand detection. In Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '11, pages 21:1--21:6, New York, NY, USA, 2011. ACM.

Digital Library

[20]

Z. Zhang, R. Alonzo, and V. Athitsos. Experiments with computer vision methods for hand detection. In Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, pages 21:1--21:6, 2011.

Digital Library

Cited By

Vasagiri K(2024)Driving Aid for Rotator Cuff Injured Patients using Hand Gesture RecognitionWSEAS TRANSACTIONS ON SIGNAL PROCESSING10.37394/232014.2024.20.320(20-31)Online publication date: 13-May-2024
https://doi.org/10.37394/232014.2024.20.3
Huang JChouvatut V(2024)Video-Based Sign Language Recognition via ResNet and LSTM NetworkJournal of Imaging10.3390/jimaging1006014910:6(149)Online publication date: 20-Jun-2024
https://doi.org/10.3390/jimaging10060149
Gupta RGupta KPandit CSingh P(2024)Sign Language Recognition using VGG16 and ResNet502024 11th International Conference on Computing for Sustainable Global Development (INDIACom)10.23919/INDIACom61295.2024.10498253(996-1001)Online publication date: 28-Feb-2024
https://doi.org/10.23919/INDIACom61295.2024.10498253
Show More Cited By

Index Terms

Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features
1. Information systems
  1. Information systems applications

Recommendations

Isolated Arabic Sign Language Recognition Using a Transformer-based Model and Landmark Keypoints
Pose-based approaches for sign language recognition provide light-weight and fast models that can be adopted in real-time applications. This article presents a framework for isolated Arabic sign language recognition using hand and face keypoints. We ...
Kinect-based Taiwanese sign-language recognition system

Gesture-recognition is an important component for many intelligent human---computer interaction applications. For example, a realtime sign-language recognition system would detect and interpret hand gestures. Many vision-based sign-language recognition ...
Kinect Based Dynamic Hand Gesture Recognition Algorithm Research
IHMSC '12: Proceedings of the 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 01

In dynamic gesture track recognition, gesture recognition rate has always been a problem, Because the traditional hand gesture recognition in how to separate the hand type from the surrounding environment, and then determine the palm position has been ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PETRA '14: Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments

May 2014

408 pages

ISBN:9781450327466

DOI:10.1145/2674396

Conference Chair:
Fillia Makedon
University of Texas at Arlington
,
Program Chairs:
Mark Clements
Georgia Institute of Technology
,
Catherine Pelachaud
TELECOM ParisTech, France
,
Vana Kalogeraki
Athens University of Economics and Bus
,
Ilias Maglogiannis
University of Piraeus, Greece

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

iPerform Center: iPerform Center for Assistive Technologies to Enhance Human Performance
CSE@UTA: Department of Computer Science and Engineering, The University of Texas at Arlington
HERACLEIA: HERACLEIA Human-Centered Computing Laboratory at UTA
U of Tex at Arlington: U of Tex at Arlington
NCRS: Demokritos National Center for Scientific Research
Fulbrigh, Greece: Fulbright Foundation, Greece

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

PETRA '14

Sponsor:

iPerform Center
CSE@UTA
HERACLEIA
U of Tex at Arlington
NCRS
Fulbrigh, Greece

PETRA '14: The 7th International Conference on PErvasive Technologies Related to Assistive Environments

May 27 - 30, 2014

Rhodes, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
373
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)4

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vasagiri K(2024)Driving Aid for Rotator Cuff Injured Patients using Hand Gesture RecognitionWSEAS TRANSACTIONS ON SIGNAL PROCESSING10.37394/232014.2024.20.320(20-31)Online publication date: 13-May-2024
https://doi.org/10.37394/232014.2024.20.3
Huang JChouvatut V(2024)Video-Based Sign Language Recognition via ResNet and LSTM NetworkJournal of Imaging10.3390/jimaging1006014910:6(149)Online publication date: 20-Jun-2024
https://doi.org/10.3390/jimaging10060149
Gupta RGupta KPandit CSingh P(2024)Sign Language Recognition using VGG16 and ResNet502024 11th International Conference on Computing for Sustainable Global Development (INDIACom)10.23919/INDIACom61295.2024.10498253(996-1001)Online publication date: 28-Feb-2024
https://doi.org/10.23919/INDIACom61295.2024.10498253
Xu XMeng KChen CLu L(2023)Isolated Word Sign Language Recognition Based on Improved SKResNet-TCN NetworkJournal of Sensors10.1155/2023/95039612023(1-10)Online publication date: 4-Jul-2023
https://doi.org/10.1155/2023/9503961
Khellas KSeghir R(2023)Alabib-65: A Realistic Dataset for Algerian Sign Language RecognitionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/359690922:6(1-23)Online publication date: 10-May-2023
https://dl.acm.org/doi/10.1145/3596909
Gao YLi YLiu BTian YZhu MLuo J(2023)Modal Fusion Sign Language Recognition Based on Swin Transformer Architecture2023 8th International Conference on Image, Vision and Computing (ICIVC)10.1109/ICIVC58118.2023.10270259(107-113)Online publication date: 27-Jul-2023
https://doi.org/10.1109/ICIVC58118.2023.10270259
Arvanitis NSartinas EKosmopoulos D(2023)Procrustes-DTW: Dynamic Time Warping Variant for the Recognition of Sign Language Utterances2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW)10.1109/ICASSPW59220.2023.10193012(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSPW59220.2023.10193012
Singh SChaturvedi A(2023)Applying Machine Learning for American Sign Language Recognition: A Brief SurveyCommunication and Intelligent Systems10.1007/978-981-99-2322-9_22(297-309)Online publication date: 11-Jul-2023
https://doi.org/10.1007/978-981-99-2322-9_22
Mistree KThakor DBhatt B(2022)A Machine Translation System from Indian Sign Language to English TextInternational Journal of Information Technologies and Systems Approach10.4018/IJITSA.31341915:1(1-23)Online publication date: 1-Jan-2022
https://doi.org/10.4018/IJITSA.313419
Al Farid FHashim NAbdullah JBhuiyan MShahida Mohd Isa WUddin JHaque MHusen M(2022)A Structured and Methodological Review on Vision-Based Hand Gesture Recognition SystemJournal of Imaging10.3390/jimaging80601538:6(153)Online publication date: 26-May-2022
https://doi.org/10.3390/jimaging8060153
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents