research-article

Public Access

Deaf and hard-of-hearing users' preferences for hearing speakers' behavior during technology-mediated in-person and remote conversations

Authors:

Matt HuenerfauthAuthors Info & Claims

W4A '21: Proceedings of the 18th International Web for All Conference

Article No.: 25, Pages 1 - 12

https://doi.org/10.1145/3430263.3452430

Published: 20 May 2021 Publication History

Abstract

Various technologies mediate synchronous audio-visual one-on-one communication (SAVOC) between Deaf and Hard-of-Hearing (DHH) and hearing colleagues, including automatic-captioning smartphone apps for in-person settings, or text-chat features of videoconferencing software in remote settings. Speech and non-verbal behaviors of hearing speakers, e.g. speaking too quietly, can make SAVOC difficult for DHH users, but prior work had not examined technology-mediated contexts. In an in-person study (N=20) with an automatic captioning smartphone app, variations in a hearing actor's enunciation and intonation dynamics affected DHH users' satisfaction. In a remote study (N=23) using a videoconferencing platform with text chat, variations in speech rate, voice intensity, enunciation, intonation dynamics, and eye contact led to such differences. This work contributes empirical evidence that specific behaviors of hearing speakers affect the accessibility of technology-mediated SAVOC for DHH users, providing motivation for future work on detecting or encouraging useful communication behaviors among hearing individuals.

References

[1]

Altieri NA, Pisoni DB, Townsend JT. Some normative data on lip-reading skills (L). J Acoust Soc Am. 2011;130(1):1--4.

[2]

S. Antia, D. Sabers, M. Stinson. 2007. Reliability and validity of the classroom communication questionnaire. Journal of Deaf Studies and Deaf Education. 12, 2, (March 2007), 158--171.

[3]

Jon P. Barker, Ricard Marxer, Emmanuel Vincent, Shinji Watanabe. 2017. The CHiME challenges: Robust speech recognition in everyday environments. In: Watanabe S., Delcroix M., Metze F., Hershey J. (eds.), New Era for Robust Speech Recognition. Springer, Cham, 327--344.

[4]

Larwan Berke, Christopher Caulfield, and Matt Huenerfauth. 2017. Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 155--164.

Digital Library

[5]

Debra L Blackwell, Jacqueline W Lucas, and Tainya C Clarke. 2014. Summary Health Statistics for U.S. Adults: National Health Interview Survey, 2012. National Center for Health Statistics. https://www.ncbi.nlm.nih.gov/ /24819891.

[6]

Denis Burnham, Sebastian Joeffry, Lauren Rice. 2010. Computer- and Human-Directed Speech Before and After Correction. In Proceedings of the 9th Speech Science and Technology Conference 2010, Melbourne, Australia. Australian Speech Science and Technology Association.

[7]

Herbert H. Clark, Meredyth A. Krych, Speaking while monitoring addressees for understanding, Journal of Memory and Language, Volume 50, Issue 1,2004, Pages 62--81, ISSN 0749-596X

[8]

Lisa B. Elliot, Michael Stinson, Syed Ahmed, and Donna Easton. 2017. User Experiences When Testing a Messaging App for Communication Between Individuals Who Are Hearing and Deaf or Hard of Hearing. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, NY, NY, USA, 405--406.

Digital Library

[9]

Lisa B. Elliot, Michael Stinson, James Mallory, Donna Easton, and Matt Huenerfauth. 2016. Deaf and Hard of Hearing Individuals' Perceptions of Communication with Hearing Colleagues in Small Groups. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 271--272.

Digital Library

[10]

C. Garberoglio, S. Cawthon, M. Bond. 2016. Deaf People and Employment in the United States. Retrieved August 4, 2017 from https://www.nationaldeafcenter.org/sites/default/files/resources/Deaf%20Employment%20Report_final.pdf.

[11]

Carrie Lou Garberoglio, Stephanie Cawthon, and Adam Sales. 2017. Deaf People and Educational Attainment in the United States: 2017. Technical Report. Washington, DC.

[12]

Google Live Transcribe. 2019. Introducing Live Transcribe. Retrieved May 7, 2020 from https://www.android.com/accessibility/live-transcribe/

[13]

Google Meet. Retrieved May 7, 2020. https://meet.google.com/

[14]

Benjamin M. Gorman and David R. Flatla. 2018. MirrorMirror: A Mobile Application to Improve Speechreading Acquisition. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, Paper 26, 1--12.

Digital Library

[15]

Gournaris, M. J., & Leigh, I. W. (2019). Comparison of Face-to-Face and Video-Mediated Communication with Deaf Individuals: Implications for Telepsychotherapy. JADARA, 37(2). Retrieved from https://repository.wcsu.edu/jadara/vol37/iss2/5

[16]

Jennifer Harris & Claire Bamford (2001) The Uphill Struggle: Services for Deaf and hard of hearing people - issues of equality, participation and access, Disability & Society, 16:7, 969--979

[17]

Hearing Loss Association of America. 2017. Basic Facts About Hearing Loss. Retrieved December 17, 2017 from http://www.hearingloss.org/content/basic-facts-about-hearing-loss.

[18]

Hnath-Chisolm T, Boothroyd A. Speechreading enhancement by voice fundamental frequency: the effects of Fo contour distortions. J Speech Hear Res. 1992;35(5):1160--1168.

[19]

IJsseldijk, F. J. (1992). Speechreading performance under different conditions of video image, repetition, and speech rate. Journal of Speech & Hearing Research, 35(2), 466--471.

[20]

Dhruv Jain, Audrey Desjardins, Leah Findlater, and Jon E. Froehlich. 2019. Autoethnography of a Hard of Hearing Traveler. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '19). Association for Computing Machinery, New York, NY, USA, 236--248.

Digital Library

[21]

Saba Kawas, George Karalis, Tzu Wen, and Richard E. Ladner. 2016. Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 15--23.

Digital Library

[22]

Konstantinidis, B. and Fels, D. (2006). Hand waving apparatus for effective turn-taking (HWAET) using video conferencing for deaf people, in the Proceedings of the 3rd Cambridge Workshop on Universal Access and Assistive Technology (CWUAAT 2006), 10--12 April 2006, Cambridge, U.K., pp. 77--82. Cambridge, U.K.: Cambridge University Press.

[23]

Raja Kushalnagar, Walter Lasecki, Jeffrey Bigham. 2014. Accessibility evaluation of classroom captions. ACM Trans. Access. Comput. 5, 3, Article 7 (Jan. 2014), 24 pp.

Digital Library

[24]

William Lewis. 2015. Skype Translator: Breaking Down Language and Hearing Barriers. Proceedings of the 37th Conference Translating and the Computer, pages 58--65, London, UK, November 26--27, 2015

[25]

Lezzoni LI, O'Day BL, Killeen M, et al. Communicating about Health Care: Observations from Persons Who Are Deaf or Hard of Hearing. Ann Intern Med. 2004;140:356--362.

[26]

J. Mallory, M. Stinson, L. Elliot, D. Easton. 2017. Personal perspectives on using automatic speech recognition to facilitate communication between deaf students and hearing customers. To appear in Proceedings of the 19th Annual International Conference on Computers and Accessibility (ASSETS '17), ACM, New York, NY.

[27]

A. O'Donnell. 2006. The role of peers and group learning. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology. 781--802. Mahwah, NJ: Lawrence Erlbaum Assoc.

[28]

Sharon Oviatt, Gina-Anne Levow, Elliott Moreton, and Margaret MacEachern. 1998. Modeling Global and Focal Hyperarticulation during Human-Computer Error Resolution. J. Acoust. Soc. Amer. 104, 3080--3098.

[29]

Carol Padden, Tom Humphries, and Carol Padden. 2009. Inside deaf culture. Harvard University Press.

[30]

Picheny MA, Durlach NI, Braida LD. Speaking clearly for the hard of hearing I: Intelligibility differences between clear and conversational speech. J Speech Hear Res. 1985 Mar; 28(1):96--103

[31]

Picheny MA, Durlach NI, Braida LD. Speaking clearly for the hard of hearing. II: Acoustic characteristics of clear and conversational speech. J Speech Hear Res. 1986 Dec; 29(4):434--4.

[32]

Picheny MA, Durlach NI, Braida LD. Speaking clearly for the hard of hearing. III: An attempt to determine the contribution of speaking rate to differences in intelligibility between clear and conversational speech. J Speech Hear Res. 1989 Sep; 32(3):600--3

[33]

Anne Marie Piper and James D. Hollan. 2008. Supporting medical conversations between deaf and hearing individuals with tabletop displays. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (CSCW '08). Association for Computing Machinery, New York, NY, USA, 147--156.

Digital Library

[34]

Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson, and Matt Huenerfauth. 2018. Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '18). Association for Computing Machinery, New York, NY, USA, 68--80.

Digital Library

[35]

Matthew Seita and Matt Huenerfauth. 2020. Deaf Individuals' Views on Speaking Behaviors of Hearing Peers when Using an Automatic Captioning App. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--8.

Digital Library

[36]

Rein Ove Sikveland. 2006. How do We Speak to Foreigners? --- Phonetic Analyses of Speech Communication between L1 and L2 Speakers of Norwegian. Working Papers 52, 109--112. Centre for Language and Literature, Lund University, Sweden.

[37]

Shen J, Souza PE. On Dynamic Pitch Benefit for Speech Recognition in Speech Masker. Front Psychol. 2018;9:1967. Published 2018 Oct 22.

[38]

Amanda J. Stent, Marie K. Huffman, and Susan E. Brennan. 2008. Adapting Speaking After Evidence of Misrecognition: Local and Global Hyperarticulation. Speech Commun. 50, 3 (March 2008), 163--178.

Digital Library

[39]

Allan Sutherland & Tessa Padden (1999) Videoconferencing for Deaf people: a case study of on-line education for Deaf people, Deafness & Education International, 1:2, 114--120

[40]

Tye-Murray, N., & Clark, W. (1998). Foundations of aural rehabilitation: Children, adults, and their family members. San Diego: Singular Pub. Group.

[41]

M. Uther, M.A. Knoll, D. Burnham, Do you speak E-NG-L-I-SH? A comparison of foreigner- and infant-directed speech, Speech Communication, Volume 49, Issue 1, 2007, Pages 2--7, ISSN 0167-6393

Digital Library

[42]

Vogler, C., Tucker, P., & Williams, N. (2013). Mixed local and remote participation in teleconferences from a deaf and hard of hearing perspective. ASSETS '13.

[43]

Gerard G Walter. 2010. Deaf and Hard-of-Hearing Students in Transition: Demographics with an Emphasis on STEM Education. National Technical Institute for the Deaf, Rochester, NY.

[44]

Emily Q. Wang and Anne Marie Piper. 2018. Accessibility in Action: Co-Located Collaboration among Deaf and Hearing Professionals. Proc. ACM Hum.-Comput Interact. 2, CSCW, Article 180 (November 2018), 25 pages.

Digital Library

[45]

Puisan Wong and Kelly Wing Sum Ng. Testing the Hyperarticulation and Prosodic Hypotheses of Child-Directed Speech: Insights From the Perceptual and Acoustic Characteristics of Child-Directed Cantonese Tones. Journal of Speech Language and Hearing Research. Volume 61, Issue 8. August 2018. Pages: 1907--1925.

[46]

Zoom. Retrieved May 7, 2020. https://zoom.us/

Cited By

McDonnell EFindlater L(2024)Envisioning Collective Communication Access: A Theoretically-Grounded Review of Captioning Literature from 2013-2023Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675649(1-18)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675649
Das MStangl AFindlater L(2024)"That comes with a huge career cost:" Understanding Collaborative Ideation Experiences of Disabled ProfessionalsProceedings of the ACM on Human-Computer Interaction10.1145/36410188:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641018
Samaradivakara YUshan TPathirage ASasikumar PKarunanayaka KKeppitiyagama CNanayakkara S(2024)SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational SettingsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650974(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650974
Show More Cited By

Index Terms

Deaf and hard-of-hearing users' preferences for hearing speakers' behavior during technology-mediated in-person and remote conversations
1. Human-centered computing
  1. Accessibility
    1. Accessibility design and evaluation methods
    2. Empirical studies in accessibility

Recommendations

Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility

Deaf and hard of hearing (DHH) individuals face barriers to communication in small-group meetings with hearing peers; we examine generation of captions on mobile devices by automatic speech recognition (ASR). While ASR output displays errors, we study ...
Deaf, Hard of Hearing, and Hearing Perspectives on Using Automatic Speech Recognition in Conversation
ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

This experience report describes the accessibility challenges in using the top seven most popular Automatic Speech Recognition (ASR) applications on personal devices for commands and group conversation, by five deaf, hard of hearing and hearing ...
How people who are deaf, Deaf, and hard of hearing use technology in creative sound activities
ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility

Creative sound activities, such as music playing and audio engineering, are said to have been democratized with the development of technology. Yet, the use of technology in creative sound activities by people who are deaf, Deaf, and hard of hearing (DHH)...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

W4A '21: Proceedings of the 18th International Web for All Conference

April 2021

224 pages

ISBN:9781450382120

DOI:10.1145/3430263

General Chairs:
Silvia Rodriguez Vazquez
University of Geneva, Switzerland
,
Ted Drake
Intuit
,
Program Chairs:
Dragan Ahmetovic
University of Milan, Italy
,
Victoria Yaneva
NBME and University of Wolverhampton, UK

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

W4A '21

W4A '21: 18th Web for All Conference

April 19 - 20, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 171 of 371 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
667
Total Downloads

Downloads (Last 12 months)248
Downloads (Last 6 weeks)29

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

McDonnell EFindlater L(2024)Envisioning Collective Communication Access: A Theoretically-Grounded Review of Captioning Literature from 2013-2023Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675649(1-18)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675649
Das MStangl AFindlater L(2024)"That comes with a huge career cost:" Understanding Collaborative Ideation Experiences of Disabled ProfessionalsProceedings of the ACM on Human-Computer Interaction10.1145/36410188:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3641018
Samaradivakara YUshan TPathirage ASasikumar PKarunanayaka KKeppitiyagama CNanayakkara S(2024)SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational SettingsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650974(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650974
Chen SWaller JSeita MVogler CKushalnagar RWang Q(2024)Towards Co-Creating Access and Inclusion: A Group Autoethnography on a Hearing Individual's Journey Towards Effective Communication in Mixed-Hearing Ability Higher Education SettingsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642017(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642017
Davis JWang HChilana PYang X(2023)"It's Not an Issue of Malice, but of Ignorance"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109017:3(1-31)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610901
Chen XLi SLiu SFowler RWang X(2023)MeetScript: Designing Transcript-based Interactions to Support Active Participation in Group Video MeetingsProceedings of the ACM on Human-Computer Interaction10.1145/36101967:CSCW2(1-32)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610196
Mittal AGupta MPoddar RNaik TKuppuraj SFogarty JKumar PJain M(2023)Jod: Examining Design and Implementation of a Videoconferencing Platform for Mixed Hearing GroupsProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608382(1-18)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608382
McDonnell E(2023)Understanding Social and Environmental Factors to Enable Collective Access Approaches to the Design of Captioning TechnologyACM SIGACCESS Accessibility and Computing10.1145/3584732.3584735(1-1)Online publication date: 15-Feb-2023
https://dl.acm.org/doi/10.1145/3584732.3584735
Dai JMoffatt K(2023)Enriching Social Sharing for the Dementia Community: Insights from In-Person and Online Social ProgramsACM Transactions on Accessible Computing10.1145/358255816:1(1-33)Online publication date: 29-Mar-2023
https://dl.acm.org/doi/10.1145/3582558
Nacimiento-García EGonzález-González CGutiérrez-Vela F(2023)Automatic captions on video calls: a must for the older adultsUniversal Access in the Information Society10.1007/s10209-023-01048-023:1(75-98)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/s10209-023-01048-0
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten