research-article

NavStick: Making Video Games Blind-Accessible via the Ability to Look Around

Authors:

Samuel Silverman,

Faizan Jamil, and

Brian A. SmithAuthors Info & Claims

UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology

October 2021

Pages 538 - 551

https://doi.org/10.1145/3472749.3474768

Published: 12 October 2021 Publication History

Abstract

Video games remain largely inaccessible to visually impaired people (VIPs). Today’s blind-accessible games are highly simplified renditions of what sighted players enjoy, and they do not give VIPs the same freedom to look around and explore game worlds on their own terms. In this work, we introduce NavStick, an audio-based tool for looking around within virtual environments, with the aim of making 3D adventure video games more blind-accessible. NavStick repurposes a game controller’s thumbstick to allow VIPs to survey what is around them via line-of-sight. In a user study, we compare NavStick with traditional menu-based surveying for different navigation tasks and find that VIPs were able to form more accurate mental maps of their environment with NavStick than with menu-based surveying. In an additional exploratory study, we investigate NavStick in the context of a representative 3D adventure game. Our findings reveal several implications for blind-accessible games, and we close by discussing these.

References

[1]

Dragan Ahmetovic, Cole Gleason, Chengxiong Ruan, Kris Kitani, Hironobu Takagi, and Chieko Asakawa. 2016. NavCog: A Navigational Cognitive Assistant for the Blind. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI ’16. ACM Press, Florence, Italy, 90–99. https://doi.org/10.1145/2935334.2935361

Digital Library

[2]

Ronny Andrade, Melissa J. Rogerson, Jenny Waycott, Steven Baker, and Frank Vetere. 2019. Playing Blind: Revealing the World of Gamers with Visual Impairment. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, 1–14. https://doi.org/10.1145/3290605.3300346

Digital Library

[3]

Ronny Andrade, Melissa J. Rogerson, Jenny Waycott, Steven Baker, and Frank Vetere. 2020. Introducing the Gamer Information-Control Framework: Enabling Access to Digital Games for People with Visual Impairment. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, 1–14. https://doi.org/10.1145/3313831.3376211

Digital Library

[4]

Dominique Archambault, Thomas Gaudy, Klaus Miesenberger, Stéphane Natkin, and Rolland Ossmann. 2008. Towards Generalised Accessibility of Computer Games. Lecture Notes in Computer Science, Vol. 5093. Springer Berlin Heidelberg, 518–527. https://doi.org/10.1007/978-3-540-69736-7_55

Digital Library

[5]

Bungie. 2004. Halo 2. Microsoft Game Studios.

[6]

Doug Church. 1999. Formal Abstract Design Tools. Gamasutra (Jul 1999). https://doi.org/10.1371/journal.pone.0213272

[7]

Doug Church. 1999. Formal Abstract Design Tools. Game Developer Magazine 6, 7 (Aug 1999), 28.

[8]

Doug Church. 2006. Formal Abstract Design Tools. MIT Press, Chapter Formal Abstract Design Tools, 366–381.

[9]

Antoine Coutrot, Sophie Schmidt, Lena Coutrot, Jessica Pittman, Lynn Hong, Jan M. Wiener, Christoph Hölscher, Ruth C. Dalton, Michael Hornberger, and Hugo J. Spiers. 2019. Virtual navigation tested on a mobile app is predictive of real-world wayfinding navigation performance. PLOS ONE 14, 3 (Mar 2019), e0213272. https://doi.org/10.1371/journal.pone.0213272

[10]

Rudolph P. Darken and John L. Sibert. 1996. Wayfinding strategies and behaviors in large virtual worlds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’96). Association for Computing Machinery, 142–149. https://doi.org/10.1145/238386.238459

[11]

Mary Beatrice Dias, Ermine A Teves, George J Zimmerman, Hend K Gedawy, Sarah M Belousov, and M Bernardine Dias. 2015. Indoor Navigation Challenges for Visually Impaired People. CRC Press, 141–164. https://doi.org/10.1201/b18220

[12]

Naughty Dog. 2016. Uncharted 4: A Thief’s End. Sony Interactive Entertainment.

[13]

Naughty Dog. 2020. The Last of Us Part II. Sony Interactive Entertainment.

[14]

Naughty Dog and Sony Interactive Entertainment. 2020. The Last of Us Part II: Accessibility Features. https://www.playstation.com/en-us/games/the-last-of-us-part-ii-ps4/accessibility/

[15]

Pierre Dognin, Igor Melnyk, Youssef Mroueh, Inkit Padhi, Mattia Rigotti, Jarret Ross, Yair Schiff, Richard A. Young, and Brian Belgodere. 2020. Image Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge. arxiv:2012.11696 [cs.CV]

[16]

Nintendo EAD. 2015. Splatoon. Nintendo.

[17]

Nintendo EPD and Monolith Soft. 2017. The Legend of Zelda: Breath of the Wild. Nintendo.

[18]

Navid Fallah, Ilias Apostolopoulos, Kostas Bekris, and Eelke Folmer. 2013. Indoor Human Navigation Systems: A Survey. Interacting with Computers 25, 1 (2013), 21–33.

[19]

Nyanchan Games. 2016. Bokurano Daibouken 3. http://www.blackscreengaming.com/bk3/overview/index.php

[20]

Reginald G. Golledge (Ed.). 1999. Wayfinding Behavior: Cognitive Mapping and Other Spatial Processes. Johns Hopkins University Press, Baltimore.

[21]

Google, Inc.2013. Google Cloud Text-to-Speech. https://cloud.google.com/text-to-speech

[22]

Isabela Granic, Adam Lobel, and Rutger C.M.E. Engels. 2014. The benefits of playing video games. American Psychologist 69, 1 (2014), 66–78.

[23]

Giles Hamilton-Fletcher, Marianna Obrist, Phil Watten, Michele Mengucci, and Jamie Ward. 2016. ”I Always Wanted to See the Night Sky”: Blind User Preferences for Sensory Substitution Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, San Jose California USA, 2162–2174. https://doi.org/10.1145/2858036.2858241

Digital Library

[24]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology. Vol. 52. Elsevier, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

[25]

Simon Holland, David R. Morse, and Henrik Gedenryd. 2002. AudioGPS: Spatial Audio Navigation with a Minimal Attention Interface. Personal and Ubiquitous Computing 6, 4 (Sept. 2002), 253–259. https://doi.org/10.1007/s007790200025

Digital Library

[26]

Robin Hunicke, Marc LeBlanc, and Robert Zubek. 2004. MDA: A Formal Approach to Game Design and Game Research. In Proceedings of the AAAI Workshop on Challenges in Game AI, Vol. 4. 1722.

[27]

Jeremy Kaldobsky. 2011. Swamp. https://www.kaldobsky.com/ssl/audiogames.php

[28]

Marc LeBlanc. 2008. 8KindsOfFun.Com: The Collected Game Design Rants of Marc ”MAHK” LeBlanc. http://algorithmancy.8kindsoffun.com/.

[29]

Masaki Matsuo, Takahiro Miura, Masatsugu Sakajiri, Junji Onishi, and Tsukasa Ono. 2016. Audible Mapper & ShadowRine: Development of Map Editor Using only Sound in Accessible Game for Blind Users, and Accessible Action RPG for Visually Impaired Gamers. In Computers Helping People with Special Needs(Lecture Notes in Computer Science), Klaus Miesenberger, Christian Bühler, and Petr Penaz(Eds.). Springer International Publishing, 537–544. https://doi.org/10.1007/978-3-319-41264-1_73

[30]

Microsoft Research. 2018. Microsoft Soundscape - Microsoft Research. https://www.microsoft.com/en-us/research/product/soundscape/.

[31]

David Molloy and Paul Carter. 2020. Last of Us Part II: Is this the most accessible game ever?BBC News (Jun 2020). https://www.bbc.com/news/technology-53093613

[32]

Vishnu Nair and Brian A. Smith. 2020. Toward Self-Directed Navigation for People with Visual Impairments. In Adjunct Publication of the 33rd Annual ACM Symposium on User Interface Software and Technology(UIST ’20 Adjunct). Association for Computing Machinery, 139–141. https://doi.org/10.1145/3379350.3416156

Digital Library

[33]

Out of Sight Games. 2017. A Hero’s Call.

[34]

Pin Interactive. 2003. Terraformers.

[35]

John R. Porter and Julie A. Kientz. 2013. An empirical study of issues and barriers to mainstream video game accessibility. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’13). Association for Computing Machinery, 1–8. https://doi.org/10.1145/2513383.2513444

Digital Library

[36]

Daisuke Sato, Uran Oh, Kakuya Naito, Hironobu Takagi, Kris Kitani, and Chieko Asakawa. 2017. NavCog3: An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS ’17. ACM Press, Baltimore, Maryland, USA, 270–279. https://doi.org/10.1145/3132525.3132535

Digital Library

[37]

Brian A. Smith and Shree K. Nayar. 2018. The RAD: Making Racing Games Equivalently Accessible to People Who Are Blind. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, Montreal QC, Canada, 1–12. https://doi.org/10.1145/3173574.3174090

Digital Library

[38]

Gearbox Software. 2019. Borderlands 3. 2K Games.

[39]

Unity Technologies. 2018. 3D Game Kit. https://assetstore.unity.com/packages/templates/tutorials/3d-game-kit-115747/

[40]

Shari Trewin, Vicki L. Hanson, Mark R. Laff, and Anna Cavender. 2008. PowerUp: an accessible virtual world. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility - Assets ’08. ACM Press, 177–184. https://doi.org/10.1145/1414471.1414504

Digital Library

[41]

Unity Technologies. 2020. Unity. Unity Technologies.

[42]

Valve Software. 2019. Steam Audio. https://valvesoftware.github.io/steam-audio/

[43]

Steve Watts. 2021. 2020 Video Game Accessibility Awards Announced. GameSpot (Mar 2021). https://www.gamespot.com/articles/2020-video-game-accessibility-awards-announced/1100-6484476/

[44]

T. Westin. 2004. Game Accessibility Case Study: Terraformers – a Real-Time 3d Graphic Game. In In Proc. of the The Fifth International Conference on Disability, Virtual Reality and Associated Technologies. 95–100.

[45]

Jeff Wilson, Bruce N. Walker, Jeffrey Lindsay, Craig Cambias, and Frank Dellaert. 2007. SWAN: System for Wearable Audio Navigation. In 2007 11th IEEE International Symposium on Wearable Computers. IEEE, Boston, MA, USA, 1–8. https://doi.org/10.1109/ISWC.2007.4373786

Digital Library

[46]

Rayoung Yang, Sangmi Park, Sonali R. Mishra, Zhenan Hong, Clint Newsom, Hyeon Joo, Erik Hofer, and Mark W. Newman. 2011. Supporting Spatial Awareness and Independent Wayfinding for Pedestrians with Visual Impairments. In The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS ’11. ACM Press, Dundee, Scotland, UK, 27. https://doi.org/10.1145/2049536.2049544

[47]

Bei Yuan, Eelke Folmer, and Frederick C. Harris. 2011. Game accessibility: a survey. Universal Access in the Information Society 10, 1 (Mar 2011), 81–100. https://doi.org/10.1007/s10209-010-0189-5

Digital Library

[48]

Shengdong Zhao, Pierre Dragicevic, Mark Chignell, Ravin Balakrishnan, and Patrick Baudisch. 2007. earPod: Eyes-Free Menu Selection Using Touch Input and Reactive Audio Feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’07. ACM Press, San Jose, California, USA, 1395–1404. https://doi.org/10.1145/1240624.1240836

Digital Library

Cited By

Chang RHung CChen BJain DGuo A(2024)SoundShift: Exploring Sound Manipulations for Accessible Mixed-Reality AwarenessDesigning Interactive Systems Conference10.1145/3643834.3661556(116-132)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661556
Espeseth ARaaen K(2024)SonicPlay: A demo of games played using only soundProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3661084(426-428)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3661084
Nair VZhu HSong PWang JSmith B(2024)Surveyor: Facilitating Discovery Within Video Games for Blind and Low Vision PlayersProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642615(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642615
Show More Cited By

Recommendations

Uncovering Visually Impaired Gamers’ Preferences for Spatial Awareness Tools Within Video Games
ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility

Sighted players gain spatial awareness within video games through sight and spatial awareness tools (SATs) such as minimaps. Visually impaired players (VIPs), however, must often rely heavily on SATs to gain spatial awareness, especially in complex ...
Read More
Towards a Generalized Acoustic Minimap for Visually Impaired Gamers
UIST '21 Adjunct: Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology

Video games created for visually impaired players (VIPs) remain inequivalent to those created for sighted players. Sighted players use minimaps within games to learn how their surrounding environment is laid out, but there is no effective analogue to ...
Read More
Toward Self-Directed Navigation for People with Visual Impairments
UIST '20 Adjunct: Adjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

Audio navigation tools have the potential to help visually impaired people (VIPs) navigate independently, but today's audio navigation tools merely guide users to destinations rather than give users the full freedom to explore and navigate environments ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology

October 2021

1357 pages

ISBN:9781450386357

DOI:10.1145/3472749

Editors:
Jeffrey Nichols
Apple, USA
,
Ranjitha Kumar
UIUC, USA
,
Michael Nebeling
University of Michigan, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

UIST '21

Sponsor:

UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology

October 10 - 14, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
649
Total Downloads

Downloads (Last 12 months)212
Downloads (Last 6 weeks)25

Other Metrics

View Author Metrics

Citations

Cited By

Chang RHung CChen BJain DGuo A(2024)SoundShift: Exploring Sound Manipulations for Accessible Mixed-Reality AwarenessDesigning Interactive Systems Conference10.1145/3643834.3661556(116-132)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661556
Espeseth ARaaen K(2024)SonicPlay: A demo of games played using only soundProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3661084(426-428)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3661084
Nair VZhu HSong PWang JSmith B(2024)Surveyor: Facilitating Discovery Within Video Games for Blind and Low Vision PlayersProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642615(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642615
Ribeiro RGonçalves IPiçarra MSeixas Pereira LDuarte CRodrigues AGuerreiro J(2024)Investigating Virtual Reality Locomotion Techniques with Blind PeopleProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642088(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642088
Coronado ACarvalho SBerretta L(2023)Game accessibility: Adaptation of a digital escape room game to improve spatial cognitive skills on blind peopleProceedings of the 25th Symposium on Virtual and Augmented Reality10.1145/3625008.3625039(174-182)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625008.3625039
Chheda-Kothary ARios DSmith KReyna AZhang CSmith B(2023)Understanding Blind and Low Vision Users' Attitudes Towards Spatial Interactions in Desktop Screen ReadersProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614490(1-5)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3614490
Collins JJung CJang YMontour DWon AAzenkot S(2023)“The Guide Has Your Back”: Exploring How Sighted Guides Can Enhance Accessibility in Social Virtual Reality for Blind and Low Vision PeopleProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608386(1-14)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608386
Jain GHindi BCourtien CXu XWyrick CMalcolm MSmith B(2023)Front Row: Automatically Generating Immersive Audio Representations of Tennis Broadcasts for Blind ViewersProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606830(1-17)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606830
Jain GTeng YCho DXing YAziz MSmith B(2023)"I Want to Figure Things Out": Supporting Exploration in Navigation for People with Visual ImpairmentsProceedings of the ACM on Human-Computer Interaction10.1145/35794967:CSCW1(1-28)Online publication date: 16-Apr-2023
https://dl.acm.org/doi/10.1145/3579496
Piçarra MRodrigues AGuerreiro J(2023)Evaluating Accessible Navigation for Blind People in Virtual EnvironmentsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585813(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585813
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents