research-article

Public Access

Deaf, Hard of Hearing, and Hearing Perspectives on Using Automatic Speech Recognition in Conversation

Authors:

Abraham Glasser,

Kesavan Kushalnagar,

Raja KushalnagarAuthors Info & Claims

ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

Pages 427 - 432

https://doi.org/10.1145/3132525.3134781

Published: 19 October 2017 Publication History

Abstract

This experience report describes the accessibility challenges in using the top seven most popular Automatic Speech Recognition (ASR) applications on personal devices for commands and group conversation, by five deaf, hard of hearing and hearing participants, including the authors. The report discusses the most common use cases, their challenges, and best practices plus pitfalls to avoid in using personal devices with ASR for commands or conversation.

References

[1]

Frank R. Lin, John K. Niparko, and Luigi Ferrucci. Hearing Loss Prevalence in the United States. Archives of Internal Medicine, 171(20):1851, nov 2011.

[2]

B B Blanchfield, J J Feldman, J L Dunbar, and E N Gardner. The severely to profoundly hearing-impaired population in the United States: prevalence estimates and demographics. Journal of the American Academy of Audiology, 12(4):183-9, apr 2001.

[3]

Sven L. Mattys, Matthew H. Davis, Ann R. Bradlow, and Sophie K. Scott. Speech recognition in adverse conditions: A review. Language and Cognitive Processes, 27(7-8):953-978, sep 2012.

[4]

Ross E. Mitchell, Travas A. Young, Bellamie Bachleda, and Michael A. Karchmer. How Many People Use ASL in the United States? Why Estimates Need Updating. Sign Language Studies, 6(3):306-335, 2006.

[5]

C Jeyalakshmi, V Krishnamurthi, and A Revathy. Deaf speech assessment using digital processing techniques. Signal & Image Processing: An International Journal (SIPIJ), 1(1):14-25, 2010.

[6]

Leo De Raeve. A Longitudinal Study on Auditory Perception and Speech Intelligibility in Deaf Children Implanted Younger Than 18 Months in Comparison to Those Implanted at Later Ages. Otology & Neurotology, 31(8):1261-1267, oct 2010.

[7]

Kinfe Tadesse Mengistu and Frank Rudzicz. Comparing Humans and Automatic Speech Recognition Systems in Recognizing Dysarthric Speech. pages 291-300. 2011.

Digital Library

[8]

Linda G. Gottermeier and Raja S. Kushalnagar. User Evaluation of Automatic Speech Recognition Systems for Deaf-Hearing Interactions at School and Work. Audiology Today, 28(2):20-34, 2016.

[9]

Marc Marschark, Patricia Sapere, Carol Convertino, and Jeff Pelz. Learning via direct and mediated instruction by deaf students. Journal of deaf studies and deaf education, 13(4):546-561, 2008.

[10]

Marc Marschark, Venetta Lampropoulou, and Emmanouil K Skordilis. Diversity in Deaf Education. Oxford University Press, 2016.

[11]

Raja. S. Kushalnagar, Walter S. Lasecki, and Jeffrey P. Bigham. Accessibility Evaluation of Classroom Captions. ACM Transactions on Accessible Computing, 5(3):1-24, 2014.

Digital Library

[12]

Karen L. Anderson. Access is the issue, not hearing loss: New policy clarification requires schools to ensure effective communication access. SIG 9 Perspectives on Hearing and Hearing Disorders in Childhood, 25(1):24-36, 2015.

[13]

Stephanie A. Borrie, Melissa Baese-Berk, Kristin Van Engen, and Tessa Bent. A relationship between processing speech in noise and dysarthric speech. The Journal of the Acoustical Society of America, 141(6):4660-4667, jun 2017.

[14]

Abraham Glasser, Kesavan Kushalnagar, and Raja Kushalnagar. Feasibility of Using Automatic Speech Recognition with Voices of Deaf and Hard-of-Hearing Individuals. In The 19th International ACM SIGACCESS Conference on Computers and Accessibility, pages 334-336, 2017.

Digital Library

[15]

W Xiong, J Droppo, X Huang, F Seide, M. Seltzer, Andreas Stolcke, D Yu, and G Zweig. The Microsoft 2016 conversational speech recognition system. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5255-5259. IEEE, mar 2017.

[16]

John Shinal. Making sense of Google CEO Sundar Pichai's plan to move every direction at once. CNBC, May 2017.

[17]

Ming Tu, Alan Wisler, Visar Berisha, and Julie M. Liss. The relationship between perceptual disturbances in dysarthric speech and automatic speech recognition performance. The Journal of the Acoustical Society of America, 140(5):EL416-EL422, Nov 2016.

[18]

Myungjong Kim, Younggwan Kim, Joohong Yoo, Jun Wang, and Hoirin Kim. Regularized Speaker Adaptation of KL-HMM for Dysarthric Speech Recognition. IEEE Transactions on Neural Systems and Rehabilitation Engineering, pages 1-1, 2017.

[19]

Mike Wald. Using automatic speech recognition to enhance education for all students: Turning a vision into reality. In Frontiers in Education, 2005. FIE'05. Proceedings 35th Annual Conference, pages S3G-S3G. IEEE, 2005.

[20]

Richard Kheir and Thomas Way. Inclusion of deaf students in computer science classes using real-time speech transcription. ACM Sigcse Bulletin, 39(3):261-265, 2007.

Digital Library

[21]

Wayne Xiong, Lingfeng Wu, Fil Alleva, Jasha Droppo, Xuedong Huang, and Andreas Stolcke. The microsoft 2017 conversational speech recognition system. Technical report, August 2017.

[22]

Dong Yu and Li Deng. Automatic speech recognition: A deep learning approach. Springer, 2014.

Digital Library

Cited By

Devries PTran NDelk KMiga MTaulbee RPidathala PGlasser AKushalnagar RVogler C(2024)Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651075(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651075
McDonnell EEagle TSinlapanuntakul PMoon SRingland KFroehlich JFindlater L(2024)“Caption It in an Accessible Way That Is Also Enjoyable”: Characterizing User-Driven Captioning Practices on TikTokProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642177(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642177
Tran NDeVries PSeita MKushalnagar RGlasser AVogler C(2024)Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal AssistantsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642094(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642094
Show More Cited By

Index Terms

Deaf, Hard of Hearing, and Hearing Perspectives on Using Automatic Speech Recognition in Conversation
1. Human-centered computing
  1. Accessibility
    1. Accessibility theory, concepts and paradigms
    2. Empirical studies in accessibility

Recommendations

Feasibility of Using Automatic Speech Recognition with Voices of Deaf and Hard-of-Hearing Individuals
ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

Many personal devices have transitioned from visual-controlled interfaces to speech-controlled interfaces to reduce costs and interactive friction, supported by the rapid growth in capabilities of speech-controlled interfaces, e.g., Amazon Echo or Apple'...
Cued Speech automatic recognition in normal-hearing and deaf subjects

This article discusses the automatic recognition of Cued Speech in French based on hidden Markov models (HMMs). Cued Speech is a visual mode which, by using hand shapes in different positions and in combination with lip patterns of speech, makes all the ...
Deaf and hard-of-hearing users' preferences for hearing speakers' behavior during technology-mediated in-person and remote conversations
W4A '21: Proceedings of the 18th International Web for All Conference

Various technologies mediate synchronous audio-visual one-on-one communication (SAVOC) between Deaf and Hard-of-Hearing (DHH) and hearing colleagues, including automatic-captioning smartphone apps for in-person settings, or text-chat features of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

October 2017

450 pages

ISBN:9781450349260

DOI:10.1145/3132525

General Chair:
Amy Hurst
University of Maryland, Baltimore County, USA
,
Program Chairs:
Leah Findlater
University of Washington, USA
,
Meredith Ringel Morris
Microsoft Research, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

ASSETS '17

Sponsor:

SIGACCESS

ASSETS '17: The 19th International ACM SIGACCESS Conference on Computers and Accessibility

October 20 - November 1, 2017

Maryland, Baltimore, USA

Acceptance Rates

ASSETS '17 Paper Acceptance Rate 28 of 126 submissions, 22%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
982
Total Downloads

Downloads (Last 12 months)146
Downloads (Last 6 weeks)25

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Devries PTran NDelk KMiga MTaulbee RPidathala PGlasser AKushalnagar RVogler C(2024)Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651075(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651075
McDonnell EEagle TSinlapanuntakul PMoon SRingland KFroehlich JFindlater L(2024)“Caption It in an Accessible Way That Is Also Enjoyable”: Characterizing User-Driven Captioning Practices on TikTokProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642177(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642177
Tran NDeVries PSeita MKushalnagar RGlasser AVogler C(2024)Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal AssistantsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642094(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642094
McDonnell EMoon SJiang LGoodman SKushalnagar RFroehlich JFindlater L(2023)“Easier or Harder, Depending on Who the Hearing Person Is”: Codesigning Videoconferencing Tools for Small Groups with Mixed Hearing StatusProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580809(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580809
Chand SAjay SKulkarni V(2023)From Disparity to Fairness: Analyzing Challenges and Solutions in AI Systems for Disabled Individuals2023 IEEE International Conference on Computer Vision and Machine Intelligence (CVMI)10.1109/CVMI59935.2023.10465050(1-6)Online publication date: 10-Dec-2023
https://doi.org/10.1109/CVMI59935.2023.10465050
Nasr SDuwairi RQuwaider M(2023)End-to-End Speech Recognition For Arabic DialectsArabian Journal for Science and Engineering10.1007/s13369-023-07670-748:8(10617-10633)Online publication date: 1-Mar-2023
https://doi.org/10.1007/s13369-023-07670-7
Sridhar APoddar RJain MKumar P(2023)Challenges Faced by the Employed Indian DHH CommunityHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42280-5_13(201-223)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42280-5_13
Mathew RDannels WParker A(2023)An Augmented Reality Based Approach for Optimization of Language Access Services in Healthcare for Deaf PatientsUniversal Access in Human-Computer Interaction10.1007/978-3-031-35897-5_3(29-52)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35897-5_3
Pragt Lvan Hengel PGrob DWasmann J(2022)Preliminary Evaluation of Automated Speech Recognition Apps for the Hearing Impaired and DeafFrontiers in Digital Health10.3389/fdgth.2022.8060764Online publication date: 16-Feb-2022
https://doi.org/10.3389/fdgth.2022.806076
Matias JHounsel AFeamster N(2022)Software-Supported Audits of Decision-Making Systems: Testing Google and Facebook's Political Advertising PoliciesProceedings of the ACM on Human-Computer Interaction10.1145/35129656:CSCW1(1-19)Online publication date: 7-Apr-2022
https://dl.acm.org/doi/10.1145/3512965
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten