research-article

Open access

Towards Accessible Musical Performances in Virtual Reality: Designing a Conceptual Framework for Omnidirectional Audio Descriptions

Authors:

Hamdi Korreshi,

Sooyeon LeeAuthors Info & Claims

ASSETS '24: Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility

Article No.: 6, Pages 1 - 17

https://doi.org/10.1145/3663548.3675618

Published: 27 October 2024 Publication History

All formats PDF

Abstract

Our research focuses on making musical performance experience in virtual reality (VR) settings non-visually accessible for Blind and Low Vision (BLV) individuals by designing a conceptual framework for omnidirectional audio descriptions (AD). We address BLV users’ prevalent challenges in accessing effective AD during VR musical performances. Employing a two-phased interview methodology, we initially collected qualitative data about BLV AD users’ experiences, followed by gathering insights from BLV professionals who specialize in AD. This approach ensures that the developed solutions are both user-centric and practically feasible. The study devises strategies for three design concepts of omnidirectional AD (Spatial AD, View-dependent AD, and Explorative AD) tailored to different types of musical performances, which vary in their visual and auditory components. Each design concept offers unique benefits; collectively, they enhance accessibility and enjoyment for BLV audiences by addressing specific user needs. Key insights highlight the crucial role of flexibility and user control in AD implementation. Based on these insights, we propose a comprehensive conceptual framework to enhance musical experiences for BLV users within VR environments.

References

[1]

Hojun Aan, Sangsun Han, Hyeonkyu Kim, Jimoon Kim, Pilhyoun Yoon, and Kibum Kim. 2021. Remote Virtual Showdown: A Collaborative Virtual Reality Game for People with Visual Impairments. arXiv preprint arXiv:2103.16153 (2021).

[2]

Lucas S Althoff, Alessandro R Silva, Marcelo Carvalho, and Mylene Q Farias. 2024. 360Align: An Open Dataset and Software for Investigating QoE and Head Motion in 360 Videos with Alignment Edits. In Proceedings of the 2024 ACM International Conference on Interactive Media Experiences. 41–55.

Digital Library

[3]

Ronny Andrade, Melissa J Rogerson, Jenny Waycott, Steven Baker, and Frank Vetere. 2019. Playing blind: Revealing the world of gamers with visual impairment. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[4]

Joanne Armitage and Kia Ng. 2016. Feeling sound: Exploring a haptic-audio relationship. In Music, Mind, and Embodiment: 11th International Symposium, CMMR 2015, Plymouth, UK, June 16-19, 2015, Revised Selected Papers 11. Springer, 146–152.

[5]

Mauro Avila Soto, Markus Funk, Matthias Hoppe, Robin Boldt, Katrin Wolf, and Niels Henze. 2017. Dronenavigator: Using leashed and free-floating quadcopters to navigate visually impaired travelers. In Proceedings of the 19th international acm sigaccess conference on computers and accessibility. 300–304.

Digital Library

[6]

Shiri Azenkot, Catherine Feng, and Maya Cakmak. 2016. Enabling building service robots to guide blind people a participatory design approach. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 3–10.

[7]

Nancy K Baym. 2015. Connect with your audience! The relational labor of connection. The communication review 18, 1 (2015), 14–22.

[8]

Bernd Benecke. 2004. Audio-description. Meta 49, 1 (2004), 78–80.

[9]

Cynthia L Bennett, Erin Brady, and Stacy M Branham. 2018. Interdependence as a frame for assistive technology research and design. In Proceedings of the 20th international acm sigaccess conference on computers and accessibility. 161–173.

Digital Library

[10]

Cynthia L Bennett, Cole Gleason, Morgan Klaus Scheuerman, Jeffrey P Bigham, Anhong Guo, and Alexandra To. 2021. “It’s Complicated”: Negotiating Accessibility and (Mis) Representation in Image Descriptions of Race, Gender, and Disability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.

Digital Library

[11]

William Boateng. 2012. Evaluating the efficacy of focus group discussion (FGD) in qualitative social research. International Journal of Business and Social Science 3, 7 (2012).

[12]

Aditya Bodi, Pooyan Fazli, Shasta Ihorn, Yue-Ting Siu, Andrew T Scott, Lothar Narins, Yash Kant, Abhishek Das, and Ilmi Yoon. 2021. Automated Video Description for Blind and Low Vision Users. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–7.

[13]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.

[14]

Virginia P Campos, Tiago MU de Araújo, Guido L de Souza Filho, and Luiz MG Gonçalves. 2020. CineAD: a system for automated audio description script generation for the visually impaired. Universal Access in the Information Society 19, 1 (2020), 99–111.

[15]

Virgínia P Campos, Luiz MG Gonçalves, Wesnydy L Ribeiro, Tiago MU Araújo, Thaís G Do Rego, Pedro HV Figueiredo, Suanny FS Vieira, Thiago FS Costa, Caio C Moraes, Alexandre CS Cruz, 2023. Machine Generation of Audio Description for Blind and Visually Impaired People. ACM Transactions on Accessible Computing 16, 2 (2023), 1–28.

Digital Library

[16]

Virginio Cantoni, Luca Lombardi, Alessandra Setti, Stanislav Gyoshev, Dimitar Karastoyanov, and Nikolay Stoimenov. 2018. Art masterpieces accessibility for blind and visually impaired people. In Computers Helping People with Special Needs: 16th International Conference, ICCHP 2018, Linz, Austria, July 11-13, 2018, Proceedings, Part II 16. Springer, 267–274.

[17]

John M Carroll, Sooyeon Lee, Madison Reddie, Jordan Beck, and Mary Beth Rosson. 2020. Human-Computer Synergies in Prosthetic Interactions.IxD&A 44 (2020), 29–52.

[18]

Claire Louise Castle, Karen Burland, and Alinka Greasley. 2022. Attending live music events with a visual impairment: experiences, accessibility and recommendations for the future. Arts and the Market 12, 2 (2022), 164–179.

[19]

Amelia Cavallo. 2015. Seeing the word, hearing the image: the artistic possibilities of audio description in theatrical performance. Research in Drama Education: The journal of applied theatre and performance 20, 1 (2015), 125–134.

[20]

Ruei-Che Chang, Chao-Hsien Ting, Chia-Sheng Hung, Wan-Chen Lee, Liang-Jin Chen, Yu-Tzu Chao, Bing-Yu Chen, and Anhong Guo. 2022. Omniscribe: Authoring immersive audio descriptions for 360 videos. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–14.

Digital Library

[21]

Agnieszka Chmiel and Iwona Mazur. 2022. A homogenous or heterogeneous audience? Audio description preferences of persons with congenital blindness, non-congenital blindness and low vision. Perspectives 30, 3 (2022), 552–567.

[22]

Jun Dong Cho. 2021. A study of multi-sensory experience and color recognition in visual arts appreciation of people with visual impairment. Electronics 10, 4 (2021), 470.

[23]

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).

[24]

Jazmin Collins, Crescentia Jung, Yeonju Jang, Danielle Montour, Andrea Stevenson Won, and Shiri Azenkot. 2023. “The Guide Has Your Back”: Exploring How Sighted Guides Can Enhance Accessibility in Social Virtual Reality for Blind and Low Vision People. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility. 1–14.

Digital Library

[25]

Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning. 160–167.

Digital Library

[26]

Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.

[27]

Andrew Crossan and Stephen Brewster. 2008. Multimodal trajectory playback for teaching shape information and trajectories to visually impaired computer users. ACM Transactions on Accessible Computing (TACCESS) 1, 2 (2008), 1–34.

Digital Library

[28]

Khang Dang, Hamdi Korreshi, Yasir Iqbal, and Sooyeon Lee. 2023. Opportunities for Accessible Virtual Reality Design for Immersive Musical Performances for Blind and Low-Vision People. In Proceedings of the 2023 ACM Symposium on Spatial User Interaction. 1–21.

Digital Library

[29]

Maitraye Das, Alexander J Fiannaca, Meredith Ringel Morris, Shaun K Kane, and Cynthia L Bennett. 2024. From Provenance to Aberrations: Image Creator and Screen Reader User Perspectives on Alt Text for AI-Generated Images. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–21.

Digital Library

[30]

Elena Di Giovanni. 2018. Audio description for live performances and audience participation. The Journal of Specialised Translation 29 (2018), 189–211.

[31]

Equal Entry. 2022. Audio Descriptions for 360 Degree Video: Best Practices. https://www.youtube.com/watch?v=jOX6gxUZq8w

[32]

Deborah I Fels, John Patrick Udo, Peter Ting, Jonas E Diamond, and Jeremy I Diamond. 2006. Odd Job Jack described: a universal design approach to described video. Universal Access in the Information society 5 (2006), 73–81.

[33]

Richard E Ferdig, Karl W Kosko, and Enrico Gandolfi. 2020. The use of ambisonic audio to improve presence, focus, and noticing while viewing 360 video. Journal For Virtual Worlds Research 13, 2-3 (2020).

[34]

Anita Fidyka and Anna Matamala. 2018. Audio description in 360º videos: Results from focus groups in Barcelona and Kraków. Translation Spaces 7, 2 (2018), 285–303.

[35]

Anita Fidyka and Anna Matamala. 2021. Retelling narrative in 360° videos: Implications for audio description. Translation Studies 14, 3 (2021), 298–312.

[36]

Anita Fidyka, Anna Matamala, Olga Soler Vilageliu, and Blanca Arias-Badia. 2021. Audio description in 360 content: results from a reception study. Skase Journal of Translation and Interpretation 14, 1 (2021), 14–32.

[37]

Louise Fryer. 2018. The independent audio describer is dead: Long live audio description!Journal of Audiovisual Translation 1, 1 (2018), 170–186.

[38]

Langis Gagnon, Claude Chapdelaine, David Byrns, Samuel Foucher, Maguelonne Heritier, and Vishwa Gupta. 2010. A computer-vision-assisted system for videodescription scripting. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 41–48.

[39]

Maarten Gerritse, Michael Rietzler, Christof Van Nimwegen, and Julian Frommel. 2024. The Effect of Spatial Audio on Curvature Gains in VR Redirected Walking. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–10.

Digital Library

[40]

Samir Ghosh, Yuhui Wang, William Zhou, Kelly Lin, Joshua McVeigh-Schultz, and Katherine Isbister. 2024. Designing Shared VR Tools for Spatial Scientific Sensemaking About Wildfire Evacuation. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–5.

[41]

Cole Gleason, Patrick Carrington, Cameron Cassidy, Meredith Ringel Morris, Kris M Kitani, and Jeffrey P Bigham. 2019. “It’s almost like they’re trying to hide it”: How User-Provided Image Descriptions Have Failed to Make Twitter Accessible. In The World Wide Web Conference. 549–559.

Digital Library

[42]

David Gonçalves, Manuel Piçarra, Pedro Pais, João Guerreiro, and André Rodrigues. 2023. " My Zelda Cane": Strategies Used by Blind Players to Play Visual-Centric Digital Games. In Proceedings of the 2023 CHI conference on human factors in computing systems. 1–15.

Digital Library

[43]

Ricardo E Gonzalez Penuela, Jazmin Collins, Cynthia Bennett, and Shiri Azenkot. 2024. Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–21.

Digital Library

[44]

João Guerreiro, Daisuke Sato, Dragan Ahmetovic, Eshed Ohn-Bar, Kris M Kitani, and Chieko Asakawa. 2020. Virtual navigation for blind people: Transferring route knowledge to the real-World. International Journal of Human-Computer Studies 135 (2020), 102369.

Digital Library

[45]

Lisa Anne Hendricks, Subhashini Venugopalan, Marcus Rohrbach, Raymond Mooney, Kate Saenko, and Trevor Darrell. 2016. Deep compositional captioning: Describing novel object categories without paired training data. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1–10.

[46]

James Herndon and Chancey Fleet. 2020. Audio descriptions for 360-degree video: Recap of Webinar. Equal Entry (2020).

[47]

Amit Hirway, Yuansong Qiao, and Niall Murray. 2024. A Quality of Experience and Visual Attention Evaluation for 360 videos with non-spatial and spatial audio. ACM Transactions on Multimedia Computing, Communications and Applications (2024).

[48]

Leona Holloway, Kim Marriott, Matthew Butler, and Alan Borning. 2019. Making sense of art: Access for gallery visitors with vision impairments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[49]

Jana Holsanova. 2011. How we focus attention in picture viewing, picture description, and during mental imagery. (2011).

[50]

Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang’Anthony’ Chen, Young-Ho Kim, and Amy Pavel. 2023. AVscript: Accessible Video Editing with Audio-Visual Scripts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.

Digital Library

[51]

Johan Isaksson-Daun, Tomas Jansson, and Johan Nilsson. 2024. Using Portable Virtual Reality to Assess Mobility of Blind and Low-Vision Individuals With the Audomni Sensory Supplementation Feedback. IEEE Access 12 (2024), 26222–26241.

[52]

Tiger F Ji, Brianna R Cochran, and Yuhang Zhao. 2022. Demonstration of VRBubble: Enhancing Peripheral Avatar Awareness for People with Visual Impairments in Social Virtual Reality. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–6.

[53]

Lucy Jiang, Crescentia Jung, Mahika Phutane, Abigale Stangl, and Shiri Azenkot. 2024. “It’s Kind of Context Dependent”: Understanding Blind and Low Vision People’s Video Accessibility Preferences Across Viewing Scenarios. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–20.

Digital Library

[54]

Lucy Jiang, Mahika Phutane, and Shiri Azenkot. 2023. Beyond audio description: Exploring 360 video accessibility with blind and low vision users through collaborative creation. In Proceedings of the 25th international ACM SIGACCESS conference on computers and accessibility. 1–17.

Digital Library

[55]

Qiao Jin, Yu Liu, Ye Yuan, Bo Han, Feng Qian, and Svetlana Yarosh. 2024. Virtual Reality, Real Pedagogy: A Contextual Inquiry of Instructor Practices with VR Video. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–21.

Digital Library

[56]

Maria Karam, Carmen Branje, Gabe Nespoli, Norma Thompson, Frank A Russo, and Deborah I Fels. 2010. The emoti-chair: an interactive tactile music exhibit. In CHI’10 Extended Abstracts on Human Factors in Computing Systems. 3069–3074.

[57]

Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3128–3137.

[58]

Finn Kensing and Jeanette Blomberg. 1998. Participatory design: Issues and concerns. Computer supported cooperative work (CSCW) 7 (1998), 167–185.

[59]

Kibum Kim, Xiangshi Ren, Seungmoon Choi, and Hong Z Tan. 2016. Assisting people with visual impairments in aiming at a target on a large wall-mounted display. International Journal of Human-Computer Studies 86 (2016), 109–120.

Digital Library

[60]

Varsha Koushik and Shaun Kane. 2023. Ability+ Motivation: Understanding Factors that Influence People with Cognitive Disabilities in Regularly Practicing Daily Activities. In Proceedings of the 20th International Web for All Conference. 122–133.

Digital Library

[61]

Julian Kreimeier and Timo Götzelmann. 2019. First steps towards walk-in-place locomotion and haptic feedback in virtual reality for visually impaired. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.

Digital Library

[62]

Julian Kreimeier, Pascal Karg, and Timo Götzelmann. 2020. BlindWalkVR: formative insights into blind and visually impaired people’s VR locomotion using commercially available approaches. In Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments. 1–8.

Digital Library

[63]

Elisa Kreiss, Cynthia Bennett, Shayan Hooshmand, Eric Zelikman, Meredith Ringel Morris, and Christopher Potts. 2022. Context Matters for Image Description Evaluation: Challenges for Referenceless Metrics. (2022).

[64]

Anatole Lécuyer, Pascal Mobuchon, Christine Mégard, Jérôme Perret, Claude Andriot, and J-P Colinot. 2003. HOMERE: a multimodal system for visually impaired people to explore virtual environments. In IEEE Virtual Reality, 2003. Proceedings. IEEE, 251–258.

[65]

Chaeeun Lee, Jinwook Kim, Hyeonbeom Yi, and Woohun Lee. 2024. Viewer2Explorer: Designing a Map Interface for Spatial Navigation in Linear 360 Museum Exhibition Video. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[66]

Sooyeon Lee, Nelson Daniel Troncoso Aldas, Chonghan Lee, Mary Beth Rosson, John M Carroll, and Vijaykrishnan Narayanan. 2022. AIGuide: Augmented Reality Hand Guidance in a Visual Prosthetic. ACM Transactions on Accessible Computing (TACCESS) 15, 2 (2022), 1–32.

Digital Library

[67]

Sooyeon Lee, Chien Wen Yuan, Benjamin V Hanrahan, Mary Beth Rosson, and John M Carroll. 2017. Reaching out: Investigating different modalities to help people with visual impairments acquire items. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility. 389–390.

Digital Library

[68]

Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun. 2018. Deep continuous fusion for multi-sensor 3d object detection. In Proceedings of the European conference on computer vision (ECCV). 641–656.

Digital Library

[69]

Xingyu Liu, Patrick Carrington, Xiang’Anthony’ Chen, and Amy Pavel. 2021. What Makes Videos Accessible to Blind and Visually Impaired People?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[70]

Xingyu" Bruce" Liu, Ruolin Wang, Dingzeyu Li, Xiang Anthony Chen, and Amy Pavel. 2022. CrossA11y: Identifying Video Accessibility Issues via Cross-modal Grounding. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–14.

[71]

Gerald Lee Lohse. 1997. Models of graphical perception. In Handbook of Human-Computer Interaction. Elsevier, 107–135.

[72]

Roberto Manduchi. 2012. Mobile vision as assistive technology for the blind: An experimental study. In Computers Helping People with Special Needs: 13th International Conference, ICCHP 2012, Linz, Austria, July 11-13, 2012, Proceedings, Part II 13. Springer, 9–16.

Digital Library

[73]

Carme Mangiron and Xiaochun Zhang. 2016. Game accessibility for the blind: Current overview and the potential application of audio description as the way forward. Researching audio description: New approaches (2016), 75–95.

[74]

Carme Mangiron and Xiaochun Zhang. 2022. Video games and audio description. In The Routledge handbook of audio description. Routledge, 377–390.

[75]

Noemi Mauro, Liliana Ardissono, and Federica Cena. 2022. Supporting people with autism spectrum disorders in the exploration of Pois: an inclusive recommender system. Commun. ACM 65, 2 (2022), 101–109.

Digital Library

[76]

Troy McDaniel, Lakshmie Narayan Viswanathan, and Sethuraman Panchanathan. 2013. An evaluation of haptic descriptions for audio described films for individuals who are blind. In 2013 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6.

[77]

David K McGookin and Stephen A Brewster. 2007. Graph builder: Constructing non-visual visualizations. In People and Computers XX—Engage: Proceedings of HCI 2006. Springer, 263–278.

[78]

Lotfi B Merabet and Jaime Sánchez. 2016. Development of an audio-haptic virtual interface for navigation of large-scale environments for people who are blind. In Universal Access in Human-Computer Interaction. Users and Context Diversity: 10th International Conference, UAHCI 2016, Held as Part of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016, Proceedings, Part III 10. Springer, 595–606.

[79]

Karyn Moffatt, Joanna McGrenere, Barbara Purves, and Maria Klawe. 2004. The participatory design of a sound and image enhanced daily planner for people with aphasia. In Proceedings of the SIGCHI conference on Human factors in computing systems. 407–414.

Digital Library

[80]

Annika Muehlbradt and Shaun K Kane. 2022. What’s in an ALT Tag? Exploring Caption Content Priorities through Collaborative Captioning. ACM Transactions on Accessible Computing (TACCESS) 15, 1 (2022), 1–32.

Digital Library

[81]

Vinaya Hanumant Naikar, Shwetha Subramanian, and Garreth W Tigwell. 2024. Accessibility Feature Implementation Within Free VR Experiences. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–9.

[82]

Suranga Nanayakkara, Elizabeth Taylor, Lonce Wyse, and S H Ong. 2009. An enhanced musical experience for the deaf: design and evaluation of a music display and a haptic chair. In Proceedings of the sigchi conference on human factors in computing systems. 337–346.

Digital Library

[83]

Rosiana Natalie. 2022. Cost-effective and Collaborative Methods to Author Video’s Scene Description for Blind People. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–5.

Digital Library

[84]

Rosiana Natalie, Ebrima Jarjue, Hernisa Kacorri, and Kotaro Hara. 2020. Viscene: A collaborative authoring tool for scene descriptions in videos. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility. 1–4.

Digital Library

[85]

Rosiana Natalie, Jolene Loh, Huei Suen Tan, Joshua Tseng, Ian Luke Yi-Ren Chan, Ebrima H Jarjue, Hernisa Kacorri, and Kotaro Hara. 2021. The efficacy of collaborative authoring of video scene descriptions. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1–15.

Digital Library

[86]

Rosiana Natalie, Jolene Loh, Huei Suen Tan, Joshua Tseng, Hernisa Kacorri, and Kotaro Hara. 2021. Uncovering patterns in reviewers’ feedback to scene description authors. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1–4.

Digital Library

[87]

Zheng Ning, Brianna L Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, Yapeng Tian, Yuhang Zhao, and Toby Jia-Jun Li. 2024. SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewers. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[88]

American Council of the Blind. 2023. All About Audio Description. https://adp.acb.org/ad.html

[89]

OpenAI. 2023. GPT-4. https://openai.com/product/gpt-4

[90]

Yagya Raj Pandeya, Bhuwan Bhattarai, and Joonwhoan Lee. 2021. Deep-learning-based multimodal emotion classification for music videos. Sensors 21, 14 (2021), 4927.

[91]

Emmanouela Patiniotaki. 2022. Audio Description for Dance Performances: An Artistic and Collaborative Approach. Status Quaestionis23 (2022).

[92]

Amy Pavel, Gabriel Reyes, and Jeffrey P Bigham. 2020. Rescribe: Authoring and automatically editing audio descriptions. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 747–759.

Digital Library

[93]

Beryl Plimmer, Andrew Crossan, Stephen A Brewster, and Rachel Blagojevic. 2008. Multimodal collaborative handwriting training for visually-impaired people. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 393–402.

Digital Library

[94]

Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele. 2016. Learning deep representations of fine-grained visual descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 49–58.

[95]

Aline Remael, Nina Reviers, and Reinhild Vandekerckhove. 2016. From translation studies and audiovisual translation to media accessibility: Some research trends. Target 28, 2 (2016), 248–260.

[96]

Aline Remael, Nina Reviers, and Gert Vercauteren. 2015. Pictures painted in words: ADLAB audio description guidelines.

[97]

David A Ross and Bruce B Blasch. 2000. Wearable interfaces for orientation and wayfinding. In Proceedings of the fourth international ACM conference on Assistive technologies. 193–200.

Digital Library

[98]

Mary Beth Rosson and John M Carroll. 2009. Scenario-based design. In Human-computer interaction. CRC Press, 161–180.

[99]

Elizabeth B-N Sanders, Eva Brandt, and Thomas Binder. 2010. A framework for organizing the tools and techniques of participatory design. In Proceedings of the 11th biennial participatory design conference. 195–198.

Digital Library

[100]

David W Schloerb, Orly Lahav, Joseph G Desloge, and Mandayam A Srinivasan. 2010. BlindAid: Virtual environment system for self-reliant trip planning and orientation and mobility training. In 2010 IEEE Haptics Symposium. IEEE, 363–370.

Digital Library

[101]

Alexa F Siu, Mike Sinclair, Robert Kovacs, Eyal Ofek, Christian Holz, and Edward Cutrell. 2020. Virtual reality without vision: A haptic and auditory white cane to navigate complex virtual worlds. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.

Digital Library

[102]

Stephen Smith. 2015. A sense of place: Place, culture and tourism. Tourism Recreation Research 40, 2 (2015), 220–233.

[103]

Joel Snyder and Esther Geiger. 2022. Opera and dance audio description. In The Routledge Handbook of Audio Description. Routledge, 168–182.

[104]

Luca Turchet, Travis West, and Marcelo M Wanderley. 2021. Touching the audience: musical haptic wearables for augmented and participatory live music performances. Personal and Ubiquitous Computing 25 (2021), 749–769.

Digital Library

[105]

John Patrick Udo, Bertha Acevedo, and Deborah I Fels. 2010. Horatio audio-describes Shakespeare’s Hamlet: Blind and low-vision theatre-goers evaluate an unconventional audio description strategy. British Journal of Visual Impairment 28, 2 (2010), 139–156.

[106]

John-Patrick Udo and Deborah I Fels. 2010. Enhancing the entertainment experience of blind and low-vision theatregoers through touch tours. Disability & Society 25, 2 (2010), 231–240.

[107]

Tess Van Daele, Akhil Iyer, Yuning Zhang, Jalyn C Derry, Mina Huh, and Amy Pavel. 2024. Making Short-Form Videos Accessible with Hierarchical Video Summaries. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–17.

Digital Library

[108]

Gert Vercauteren. 2014. A translational and narratological approach to audio describing narrative characters. TTR 27, 2 (2014), 71–90.

[109]

Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3156–3164.

[110]

Paul Viola, Michael Jones, 2001. Robust real-time object detection. International journal of computer vision 4, 34-47 (2001), 4.

[111]

Lakshmie Narayan Viswanathan, Troy McDaniel, Sreekar Krishna, and Sethuraman Panchanathan. 2010. Haptics in audio described movies. In 2010 IEEE International Symposium on Haptic Audio Visual Environments and Games. IEEE, 1–2.

[112]

Lakshmie Narayan Viswanathan, Troy McDaniel, and Sethuraman Panchanathan. 2011. Audio-haptic description in movies. In HCI International 2011–Posters’ Extended Abstracts: International Conference, HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part I 14. Springer, 414–418.

[113]

Yujia Wang, Wei Liang, Haikun Huang, Yongqi Zhang, Dingzeyu Li, and Lap-Fai Yu. 2021. Toward automatic audio description generation for accessible videos. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[114]

Ryan Wedoff, Lindsay Ball, Amelia Wang, Yi Xuan Khoo, Lauren Lieberman, and Kyle Rector. 2019. Virtual showdown: An accessible virtual reality game with scaffolds for youth with visual impairments. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.

Digital Library

[115]

Gareth R White, Geraldine Fitzpatrick, and Graham McAllister. 2008. Toward accessible 3D virtual environments for the blind and visually impaired. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts. 134–141.

Digital Library

[116]

Margot Whitfield and Deborah I Fels. 2013. Inclusive design, audio description and diversity of theatre experiences. The Design Journal 16, 2 (2013), 219–238.

[117]

Markus Wieland, Lauren Thevin, Albrecht Schmidt, and Tonja Machulla. 2022. Non-verbal Communication and Joint Attention Between People with and Without Visual Impairments: Deriving Guidelines for Inclusive Conversations in Virtual Realities. In Computers Helping People with Special Needs: 18th International Conference, ICCHP-AAATE 2022, Lecco, Italy, July 11–15, 2022, Proceedings, Part I. Springer, 295–304.

Digital Library

[118]

Jingyi Xie, Madison Reddie, Sooyeon Lee, Syed Masum Billah, Zihan Zhou, Chun-Hua Tsai, and John M Carroll. 2022. Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance. ACM Transactions on Computer-Human Interaction (TOCHI) 29, 4 (2022), 1–40.

Digital Library

[119]

Wai Yu, Douglas Reid, and Stephen Brewster. 2002. Web-based multimodal graphs for visually impaired people. In Universal Access and Assistive Technology: Proceedings of the Cambridge Workshop on UA and AT’02. Springer, 97–108.

[120]

Beste F Yuksel, Pooyan Fazli, Umang Mathur, Vaishali Bisht, Soo Jung Kim, Joshua Junhee Lee, Seung Jung Jin, Yue-Ting Siu, Joshua A Miele, and Ilmi Yoon. 2020. Human-in-the-loop machine learning to increase video accessibility for visually impaired and blind users. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. 47–60.

Digital Library

[121]

Monika Zabrocka. 2023. Hear the Flow: Report on the Rationale Method of Audio-Describing Dance Performances. Journal of Visual Impairment & Blindness 117, 3 (2023), 251–255.

[122]

Zhe-Xin Zhang. 2024. A Design of Interface for Visual-Impaired People to Access Visual Information from Images Featuring Large Language Models and Visual Language Models. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–4.

Digital Library

[123]

Yuhang Zhao, Edward Cutrell, Christian Holz, Meredith Ringel Morris, Eyal Ofek, and Andrew D Wilson. 2019. SeeingVR: A set of tools to make virtual reality more accessible to people with low vision. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–14.

Digital Library

Cited By

Dang KLee S(2024)Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision UsersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688492(1-5)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3688492

Index Terms

Towards Accessible Musical Performances in Virtual Reality: Designing a Conceptual Framework for Omnidirectional Audio Descriptions
1. Human-centered computing
2. Social and professional topics
  1. User characteristics
    1. People with disabilities

Recommendations

Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision Users
ASSETS '24: Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility

Virtual reality (VR), inherently reliant on spatial interaction, poses significant accessibility barriers for individuals who are blind or have low vision (BLV). Traditional audio descriptions (AD) typically provide a verbal explanation of visual ...
Opportunities for Accessible Virtual Reality Design for Immersive Musical Performances for Blind and Low-Vision People
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Blind and low-vision (BLV) individuals often face challenges in their attendance and appreciation of musical performances (e.g., concerts, musicals, opera) due to limited mobility and accessibility of visual information. However, the emergence of ...
Deploying Prerecorded Audio Description for Musical Theater Using Live Performance Tracking
Perception, Representations, Image, Sound, Music
Abstract
Audio description, an accessibility service used by blind or visually impaired individuals, provides spoken descriptions of visual content. This alternative format allows those with low or no vision the ability to access information that sighted ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '24: Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility

October 2024

1475 pages

ISBN:9798400706776

DOI:10.1145/3663548

Editors:
David Flatla
University of Guelph, CANADA
,
Faustina Hwang
University of Reading, UNITED KINGDOM
,
Tiago Guerreiro
University of Lisbon, PORTUGAL
,
Robin Brewer
University of Michigan, UNITED STATES

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2024

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ASSETS '24

Sponsor:

SIGACCESS

ASSETS '24: The 26th International ACM SIGACCESS Conference on Computers and Accessibility

October 27 - 30, 2024

NL, St. John's, Canada

Acceptance Rates

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
115
Total Downloads

Downloads (Last 12 months)115
Downloads (Last 6 weeks)60

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dang KLee S(2024)Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision UsersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688492(1-5)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3688492

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents