Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3371300.3383337acmconferencesArticle/Chapter ViewAbstractPublication Pagesw4aConference Proceedingsconference-collections
research-article

Deaf and hard-of-hearing users' prioritization of genres of online video content requiring accurate captions

Published: 20 April 2020 Publication History

Abstract

Online video is an important information source, yet its pace of growth, including user-submitted content, is so rapid that automatic captioning technologies are needed to make content accessible for people who are Deaf or Hard-of-Hearing (DHH). To support future creation of a research dataset of online videos, we must prioritize which genres of online video content DHH users believe are of greatest importance to be accurately captioned. Our first contribution is to validate that the Best-Worst Scaling (BWS) methodology is able to accurately gather judgments on this topic by conducting an in-person study with 25 DHH users, using a card-sorting methodology to rank the importance for various YouTube genres of online video to be accurately captioned. Our second contribution is to identify video genres of highest captioning importance via an online survey with 151 DHH individuals, and those participants highly ranked: News and Politics, Education, and Technology and Science.

Supplementary Material

ZIP File (a3-berke.zip)
Supplemental files.

References

[1]
John Albertini and Connie Mayer. 2011. Using Miscue Analysis to Assess Comprehension in Deaf College Readers. The Journal of Deaf Studies and Deaf Education (JDSDE) 16, 1 (2011), 35--46.
[2]
Jon P. Barker, Ricard Marxer, Emmanuel Vincent, Shinji Watanabe. 2017. The CHiME challenges: Robust speech recognition in everyday environments. In: Watanabe S., Delcroix M., Metze F., Hershey J. (eds.), New Era for Robust Speech Recognition. Springer, Cham, 327--344.
[3]
Larwan Berke, Christopher Caulfield, and Matt Huenerfauth. 2017. Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS '17). Association for Computing Machinery (ACM), New York, NY, USA, 155--164.
[4]
Denis Burnham, Jordi Robert-Ribes, and Ruth Ellison. 1998. Why captions have to be on time. In Proceedings of the International Conference on Auditory-Visual Speech Processing (Sydney, Australia) (AVSP '98). ISCA, Baixas, France, 4.
[5]
Juan Cao, Yong-Dong Zhang, Yi-Cheng Song, Zhi-Neng Chen, Xu Zhang, and Jin-Tao Li. 2009. MCG-WEBV: A benchmark dataset for web video analysis. Beijing:Institute of Computing Technology 10 (2009), 324--334.
[6]
Jiyoung Cha. 2013. Does genre type influence choice of video platform? A study of college student use of internet and television for specific video genres. Telematics and Informatics 30, 2 (2013), 189 -- 200.
[7]
C. Chapdelaine, V. Gouaillier, M. Beaulieu, and L. Gagnon. 2007. Improving video captioning for deaf and hearing-impaired people based on eye movement and attention overload. Proceedings of the SPIE 6492 (2007), 11.
[8]
Eli Cohen. 2009. Applying best-worst scaling to wine marketing. International journal of wine business research 21, 1 (2009), 8--23.
[9]
Michael Crabb, Rhianne Jones, Mike Armstrong, and Chris J. Hughes. 2015. Online News Videos: The UX of Subtitle Position. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility (Lisbon, Portugal) (ASSETS '15). Association for Computing Machinery (ACM), New York, NY, USA, 215--222.
[10]
Paul G. Curran. 2016. Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology 66 (2016), 4 -- 19.
[11]
Deborah I Fels, Daniel G Lee, Carmen Branje, and Matthew Hornburg. 2005. Emotive captioning and access to television. AMCIS 2005 Proceedings (2005), 300.
[12]
Adam Finn and Jordan J Louviere. 1992. Determining the appropriate response to evidence of public concern: the case of food safety. Journal of Public Policy & Marketing 11, 2 (1992), 12--25.
[13]
Terry N Flynn, Jordan J Louviere, Tim J Peters, and Joanna Coast. 2007. Best-worst scaling: what it can do for health care research and how to do it. Journal of health economics 26, 1 (2007), 171--189.
[14]
Terry N. Flynn and A.A. J. Marley. 2014. Best-Worst Scaling: Theory and Methods. Edward Elgar Publishing, Cheltenham, UK, 178--201.
[15]
Stephen Gulliver and George Ghinea. 2003. How level and type of deafness affect user perception of multimedia video clips. Universal Access in the Information Society (UAIS) 2, 4 (01 Nov 2003), 374--386.
[16]
Timothy J. Hazen. (2006). Automatic alignment and error correction of human generated transcripts for long speech recordings. INTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP.
[17]
Ellen S. Hibbard and Deb I. Fels. 2011. The Vlogging Phenomena: A Deaf Perspective. In Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (Dundee, Scotland, UK) (ASSETS '11). Association for Computing Machinery (ACM), New York, NY, USA, 59--66.
[18]
Yun Huang, Yifeng Huang, Na Xue, and Jeffrey P. Bigham. 2017. Leveraging Complementary Contributions of Different Workers for Efficient Crowdsourcing of Video Captions. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4617--4626.
[19]
Carl Jensema, Ralph McCann, and Scott Ramsey. 1996. Closed-Captioned Television Presentation Speed and Vocabulary. American Annals of the Deaf 141, 4 (1996), 284--292. http://www.jstor.org/stable/44401017
[20]
Carl J. Jensema, Ramalinga Sarma Danturthi, and Robert Burch. 2000a. Time Spent Viewing Captions on Television Programs. American Annals of the Deaf 145, 5 (2000), 464--468. http://www.jstor.org/stable/44393238
[21]
Carl J Jensema, Sameh El Sharkawy, Ramalinga Sarma Danturthi, Robert Burch, and David Hsu. 2000b. Eye movement patterns of captioned television viewers. American annals of the deaf 145, 3 (2000), 275--285.
[22]
A.K. Jones, D.L. Jones, G. Edwards-Jones, and P. Cross. 2013. Informing decision making in agricultural greenhouse gas mitigation policy: A Best-Worst Scaling survey of expert and farmer opinion in the sheep industry. Environmental Science & Policy 29 (2013), 46 -- 56.
[23]
Sushant Kafle and Matt Huenerfauth. 2017. Evaluating the Usability of Automatically Generated Captions for People Who Are Deaf or Hard of Hearing. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS '17). Association for Computing Machinery (ACM), New York, NY, USA, 165--174.
[24]
Svetlana Kiritchenko and Saif Mohammad. 2017. Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers. 465--470.
[25]
Svetlana Kiritchenko and Saif M. Mohammad. 2016. Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best-Worst Scaling. In Proceedings of The 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL). San Diego, California.
[26]
Thomas Ksiazek, Limor Peer, and Kevin Lessard. (2014). User engagement with online news: Conceptualizing interactivity and exploring the relationship between online news videos and user comments. New Media & Society. 18.
[27]
Raja Kushalnagar and Kesavan Kushalnagar. 2018. SubtitleFormatter: Making Subtitles Easier to Read for Deaf and Hard of Hearing Viewers on Personal Devices. In Computers Helping People with Special Needs, Klaus Miesenberger and Georgios Kouroupetroglou (Eds.). Springer International Publishing, Cham, 211--219.
[28]
Raja S. Kushalnagar, Walter S. Lasecki, and Jeffrey P. Bigham. 2013. Captions Versus Transcripts for Online Video Content. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (Rio de Janeiro, Brazil) (W4A '13). Association for Computing Machinery (ACM), New York, NY, USA, Article 32, 4 pages.
[29]
Raja S. Kushalnagar, John J. Rivera, Warrance Yu, and Daniel S. Steed. 2014. AVD-LV: An Accessible Player for Captioned STEM Videos. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (Rochester, New York, USA) (ASSETS '14). ACM, New York, NY, USA, 287--288.
[30]
Paddy Ladd. 2003. Understanding Deaf Culture: In Search of Deafhood. Multilingual Matters, Bristol, UK.
[31]
Daniel Lakens. (2017). Equivalence tests: A practical primer for t-tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355--362.
[32]
Jordan J Louviere, Terry N Flynn, and Anthony Alfred John Marley. 2015. Best-worst scaling: Theory, methods and applications. Cambridge University Press, Cambridge, MA.
[33]
Jordan J Louviere and George G Woodworth. 1990. Best-worst scaling: A model for largest difference judgments [Working Paper]. Faculty of Business, University of Alberta (1990).
[34]
Henry B. Mann and Donald R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18, 1 (03 1947), 50--60.
[35]
Marc Marschark, Harry G. Lang, and John A. Albertini. 2001. Educating deaf students: From research to practice (1 ed.). Oxford University Press, Oxford, UK.
[36]
Ichiro Maruyama, Yoshiharu Abe, Eiji Sawamura, Tetsuo Mitsuhashi, Terumasa Ehara, and Katsuhiko Shirai. 1999. Cognitive experiments on timing lag for superimposing closed captions. In Sixth European Conference on Speech Communication and Technology (EUROSPEECH'99). 575--578. https://www.isca-speech.org/archive/eurospeech_1999/e99_0575.html
[37]
Alex Molassiotis, Richard Emsley, Darren Ashcroft, Ann Caress, Jackie Ellis, Richard Wagland, Chris D. Bailey, Jemma Haines, Mari Lloyd Williams, Paul Lorigan, Jaclyn Smith, Carol Tishelman, and Fiona Blackhall. 2012. Applying Best-Worst scaling methodology to establish delivery preferences of a symptom supportive care intervention in patients with lung cancer. Lung Cancer 77, 1 (2012), 199 -- 204.
[38]
M. Montagnuolo and A. Messina. 2007. Automatic Genre Classification of TV Programmes Using Gaussian Mixture Models and Neural Networks. In Proceedings of the 18th International Workshop on Database and Expert Systems Applications (Regensburg, Germany) (DEXA). 99--103.
[39]
Jacob Nyarko and Kwaku Oppong Asante. 2015. Social Exclusion of the Deaf in Corporate Television Advertising in Ghana: A Pilot Study. Journal of Communication 6, 2 (2015), 284--295.
[40]
Hearing Loss Association of America. 2019. Hearing Loss Basics - How to tell if you have hearing loss. (2019). https://www.hearingloss.org/hearing-help/hearing-lossbasics/
[41]
Daniel M. Oppenheimer, Tom Meyvis, and Nicolas Davidenko. 2009. Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental. Social Psychology 45, 4 (2009), 867--872.
[42]
Seth Ovadia. 2004. Ratings and rankings: reconsidering the structure of values and their measurement. International Journal of Social Research Methodology 7, 5 (2004), 403--414.
[43]
Susan J. Parault and Heather M. Williams. 2010. Reading Motivation, Reading Amount, and Text Comprehension in Deaf and Hearing Adults. The Journal of Deaf Studies and Deaf Education (JDSDE) 15, 2 (2010), 120--135.
[44]
Dimitris Potoglou, Peter Burge, Terry Flynn, Ann Netten, Juliette Malley, Julien Forder, and John E. Brazier. 2011. Best-worst scaling vs. discrete choice experiments: An empirical comparison using social care data. Social Science & Medicine 72, 10 (2011), 1717 -- 1727.
[45]
Soraia Silva Prietch, Napoliana Silva de Souza, and Lucia Villela Leite Filgueiras. 2015. Application Requirements for Deaf Students to Use in Inclusive Classrooms. In Proceedings of the 7th Latin American Conference on Human Computer Interaction (Córdoba, Argentina) (CLIHC '15). Association for Computing Machinery (ACM), New York, NY, USA, Article 5, 8 pages.
[46]
Anni Rander and Peter Olaf Looms. 2010. The Accessibility of Television News with Live Subtitling on Digital Television. In Proceedings of the 8th European Conference on Interactive TV and Video (Tampere, Finland) (EuroITV '10). ACM, New York, NY, USA, 155--160.
[47]
Jonathan Rubin, Ryan Leisinger, and Gary Morin. 2014. 508 Accessible Videos-Why (and How) to Make Them. (June 2014). Retrieved April 29, 2019 from https:/digital.gov/2014/06/30/508-accessible-videos-why-and-how-to-make-them/
[48]
Hillary M Sackett, Robert Shupp, and Glynn Tonsor. 2013. Consumer perceptions of sustainable farming practices: A Best-Worst scenario. Agricultural and Resource Economics Review 42, 2 (2013), 275--290.
[49]
Brent N. Shiver and Rosalee J. Wolfe. 2015. Evaluating Alternatives for Better Deaf Accessibility to Selected Web-Based Multimedia. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility (Lisbon, Portugal) (ASSETS '15). Association for Computing Machinery (ACM), New York, NY, USA, 231--238.
[50]
Anselm Strauss and Juliet M. Corbin. 1998. Basics of qualitative research: Grounded theory procedures and techniques (2 ed.). SAGE Publications, Inc., Thousand Oaks, CA, USA.
[51]
Ba Tu Truong and C. Dorai. 2000. Automatic genre identification for content-based video categorization. In Proceedings of the 15th International Conference on Pattern Recognition (ICPR), Vol. 4. 230--233.
[52]
Máté Akos Tündik, György Szaszák, Gábor Gosztolya, and András Beke. 2018. User-centric Evaluation of Automatic Punctuation in ASR Closed Captioning. Proceedings of Interspeech 2018 (2018), 2628--2632.
[53]
Lucia Vesnic-Alujevic & Sofie VanBauwel (2014) YouTube: A Political Advertising Tool? A Case Study of the Use of YouTube in the Campaign for the European Parliament Elections, Journal of Political Marketing, 13:3, 195--212
[54]
J. Wu and M. Worring. 2012. Efficient Genre-Specific Semantic Video Indexing. IEEE Transactions on Multimedia 14, 2 (April 2012), 291--302.
[55]
Georgios N. Yannakakis and John Hallam. 2011. Ranking vs. Preference: A Comparative Study of Self-reporting. In Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction (Memphis, Tennessee, USA) (ACII '11), Sidney D'Mello, Arthur Graesser, Björn Schuller, and Jean-Claude Martin (Eds.). Springer, Berlin, Heidelberg, 437--446.
[56]
Georgios N. Yannakakis and Héctor P. Martínez. 2015. Ratings are Overrated! Frontiers in ICT 2 (2015), 13.
[57]
Norman Bradburn, Seymour Sudman, Brian Wansink. 2004. Asking questions: The definitive guide to questionnaire design-For market research, political polls, and social and health questionnaires (Rev. ed.). San Francisco, CA, US: Jossey-Bass.
[58]
Sofia Enamorado. 2019. Final CVAA and FCC Online Video Closed Captioning Rules. Retrieved on September 19, 2019, from https://www.3playmedia.com/2018/11/14/final-cvaa-and-fcc-online-video-closed-captioning-rules/
[59]
Larwan Berke, Sushant Kafle, and Matt Huenerfauth. 2018. Methods for Evaluation of Imperfect Captioning Tools by Deaf or Hard-of-Hearing Users at Different Reading Literacy Levels. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Paper 91, 12 pages.

Cited By

View all
  • (2024)“Signifier” Video Sharing Platform and Accessible Media Player for Deaf UsersProceedings of the 2024 International Conference on Information Technology for Social Good10.1145/3677525.3678655(151-157)Online publication date: 4-Sep-2024
  • (2024)Envisioning Collective Communication Access: A Theoretically-Grounded Review of Captioning Literature from 2013-2023Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675649(1-18)Online publication date: 27-Oct-2024
  • (2024)Audio Engineering by People Who Are deaf and Hard of Hearing: Balancing Confidence and LimitationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642454(1-13)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Deaf and hard-of-hearing users' prioritization of genres of online video content requiring accurate captions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    W4A '20: Proceedings of the 17th International Web for All Conference
    April 2020
    190 pages
    ISBN:9781450370561
    DOI:10.1145/3371300
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 April 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. captioning
    2. deaf and hard-of-hearing
    3. genres
    4. video

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    W4A '20
    W4A '20: 17th Web for All Conference
    April 20 - 21, 2020
    Taipei, Taiwan

    Acceptance Rates

    W4A '20 Paper Acceptance Rate 18 of 29 submissions, 62%;
    Overall Acceptance Rate 171 of 371 submissions, 46%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)85
    • Downloads (Last 6 weeks)16
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)“Signifier” Video Sharing Platform and Accessible Media Player for Deaf UsersProceedings of the 2024 International Conference on Information Technology for Social Good10.1145/3677525.3678655(151-157)Online publication date: 4-Sep-2024
    • (2024)Envisioning Collective Communication Access: A Theoretically-Grounded Review of Captioning Literature from 2013-2023Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675649(1-18)Online publication date: 27-Oct-2024
    • (2024)Audio Engineering by People Who Are deaf and Hard of Hearing: Balancing Confidence and LimitationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642454(1-13)Online publication date: 11-May-2024
    • (2024)Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-HearingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642287(1-18)Online publication date: 11-May-2024
    • (2024)Caption Royale: Exploring the Design Space of Affective Captions from the Perspective of Deaf and Hard-of-Hearing IndividualsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642258(1-17)Online publication date: 11-May-2024
    • (2024)“Caption It in an Accessible Way That Is Also Enjoyable”: Characterizing User-Driven Captioning Practices on TikTokProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642177(1-16)Online publication date: 11-May-2024
    • (2024)How Users Experience Closed Captions on Live Television: Quality Metrics Remain a ChallengeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641988(1-16)Online publication date: 11-May-2024
    • (2023)Accessibility Research in Digital Audiovisual Media: What Has Been Achieved and What Should Be Done Next?Proceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596159(94-114)Online publication date: 12-Jun-2023
    • (2022)Reading-Assistance Tools Among Deaf and Hard-of-Hearing Computing Professionals in the U.S.: Their Reading Experiences, Interests and Perceptions of Social AccessibilityACM Transactions on Accessible Computing10.1145/352019815:2(1-31)Online publication date: 19-May-2022
    • (2022)Authoring accessible media content on social networksProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544882(1-11)Online publication date: 23-Oct-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media