poster

In-cAR Gaming: Exploring the use of AR headsets to Leverage Passenger Travel Environments for Mixed Reality Gameplay

Authors:

Daniel Medeiros,

Stephen Anthony BrewsterAuthors Info & Claims

CHI EA '22: CHI Conference on Human Factors in Computing Systems Extended Abstracts

Article No.: 369, Pages 1 - 7

https://doi.org/10.1145/3491101.3519741

Published: 28 April 2022 Publication History

Abstract

Autonomous cars offer passengers a rich platform for Augmented Reality entertainment, with complex sensing that can drive passenger experiences by tracking, appropriating and altering elements of reality. This paper forms an early exploration of in-car AR games, starting with how existing game genres might work within an AR vehicle context, appropriating elements of reality (e.g. other cars) into gameplay, and altering the appearance of reality in relation to game events (e.g. augmented cracks in car windows). We discuss results from focus groups exploring an initial AR game prototype, and an informal in-car evaluation of a follow-up prototype inspired by the focus groups. Broadly, we found that participants enjoyed using AR gaming in-car, noted the immersive impact of appropriating real-world elements into gameplay, and felt that it improved their experience of the journey. We reflect on the ways in which future AR passenger experiences might take advantage of the available sensing and environment to create engaging reality-based gameplay.

References

[1]

Andrew Aultman, Spencer Dowie, and Nelly Ann Hamid. 2018. Design Heuristics for Mobile Augmented Reality Game User Interfaces. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/3170427.3188580

[2]

Alexander Baldwin, Jeanette Eriksson, and Carl Magnus Olsson. 2017. Bus Runner: Using Contextual Cues for Procedural Generation of Game Content on Public Transport. In HCI. 21–34. https://doi.org/10.1007/978-3-319-58077-7_2

[3]

Cara Bloom, Joshua Tan, Javed Ramjohn, and Lujo Bauer. 2017. Self-driving cars and data collection: Privacy perceptions of networked autonomous vehicles. In Thirteenth Symposium on Usable Privacy and Security ({SOUPS} 2017). 357–375.

[4]

Kody R. Dillman, Terrance Tin Hoi Mok, Anthony Tang, Lora Oehlberg, and Alex Mitchell. 2018. A Visual Interaction Cue Framework from Video Game Environments for Augmented Reality. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173714

[5]

K.A. Ericsson and H.A. Simon. 1993. Protocol Analysis: Verbal Reports as Data (Rev Ed.). The MIT Press.

[6]

Joseph L. Gabbard, Gregory M. Fitch, and Hyungil Kim. 2014. Behind the Glass: Driver Challenges and Opportunities for AR Automotive Applications. Proc. IEEE 102, 2 (2014), 124–136. https://doi.org/10.1109/JPROC.2013.2294642

[7]

Vertigo Games and Jaywalkers Interactive. 2021. Arizona Sunshine. https://arizona-sunshine.com/

[8]

David Goedicke, Alexandra W.D. Bremers, Hiroshi Yasuda, and Wendy Ju. 2021. XR-OOM: Mixing Virtual Driving Simulation with Real Cars and Environments Safely. Association for Computing Machinery, New York, NY, USA, 67–70. https://doi.org/10.1145/3473682.3480266

Digital Library

[9]

David Goedicke, Jamy Li, Vanessa Evers, and Wendy Ju. 2018. VR-OOM: Virtual Reality On-ROad Driving SiMulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3173574.3173739

Digital Library

[10]

Renate Haeuslschmid, Bastian Pfleging, and Florian Alt. 2016. A Design Space to Support the Development of Windshield Applications for the Car. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5076–5091. https://doi.org/10.1145/2858036.2858336

Digital Library

[11]

Philipp Hock, Sebastian Benedikter, Jan Gugenheimer, and Enrico Rukzio. 2017. CarVR: Enabling In-Car Virtual Reality Entertainment. 4034–4044. https://doi.org/10.1145/3025453.3025665

[12]

Holoride. 2021. Turning vehicles into moving theme parks. https://www.holoride.com/

[13]

Mohamed Kari, Tobias Grosse-Puppendahl, Alexander Jagaciak, David Bethge, Reinhard Schütte, and Christian Holz. 2021. SoundsRide: Affordance-Synchronized Music Mixing for In-Car Audio Augmented Reality. 118–133. https://doi.org/10.1145/3472749.3474739

[14]

Kiloo. 2021. Subway Surfers Mobile Game. https://www.kiloo.com/subway-surfers/

[15]

Matthew Lakier, Lennart E. Nacke, Takeo Igarashi, and Daniel Vogel. 2019. Cross-Car, Multiplayer Games for Semi-Autonomous Driving. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (CHI PLAY ’19). Association for Computing Machinery, New York, NY, USA, 467–480. https://doi.org/10.1145/3311350.3347166

Digital Library

[16]

Jingyi Li, Ceenu George, Andrea Ngao, Kai Holländer, Stefan Mayer, and Andreas Butz. 2020. An Exploration of Users’ Thoughts on Rear-Seat Productivity in Virtual Reality. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 92–95. https://doi.org/10.1145/3409251.3411732

Digital Library

[17]

Ford Global Technologies LLC. 2016. Autonomous Vehicle Entertainment System. Patent US9272708B2.

[18]

Mark McGill, Julie Williamson, Alexander Ng, Frank Pollick, and Stephen Brewster. 2019. Challenges in passenger use of mixed reality headsets in cars and other transportation. Virtual Reality 24, 4 (2019), 583–603. https://doi.org/10.1007/s10055-019-00420-x

Digital Library

[19]

Nintendo. 2021. Mario Kart Live: Home Circuit. https://www.nintendo.co.uk/Games/Nintendo-Switch-download-software/Mario-Kart-Live-Home-Circuit-1832413.html

[20]

Brian Normile. 2020. Which Cars Have Head-Up Displays for 2020?https://www.cars.com/articles/which-cars-have-head-up-displays-for-2020-421615/

[21]

NReal. 2020. NReal Light AR Glasses. https://www.nreal.ai/

[22]

Ohan Oda, Levi J. Lister, Sean White, and Steven Feiner. 2008. Developing an Augmented Reality Racing Game. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).

[23]

Pablo E. Paredes, Stephanie Balters, Kyle Qian, Elizabeth L. Murnane, Francisco Ordóñez, Wendy Ju, and James A. Landay. 2018. Driving with the Fishes: Towards Calming and Mindful Virtual Reality Experiences for the Car. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 184 (Dec. 2018), 21 pages. https://doi.org/10.1145/3287062

Digital Library

[24]

Annie Pauzie. 2015. Head Up Display in Automotive: A New Reality for the Driver. In Design, User Experience, and Usability: Interactive Experience Design, Aaron Marcus (Ed.). Springer International Publishing, Cham, 505–516.

[25]

Rinspeed. 2014. Rinspeed XCHANGE. https://www.rinspeed.eu/en/XchangE_24_concept-car.html

[26]

Johnny Saldaña. 2015. The Coding Manual for Qualitative Researchers. SAGE Publications.

[27]

Nadja Schomig, Katharina Wiedemann, Frederik Naujoks, Alexandra Neukum, Bettina Leuchtenberg, and Thomas Vohringer-Kuhnt. 2018. An Augmented Reality Display for Conditionally Automated Driving. Adjunct Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (2018). https://doi.org/10.1145/3239092.3265956

[28]

Hanna Kathrin Schraffenberger. 2018. Arguably augmented reality: relationships between the virtual and the real. Ph. D. Dissertation. Leiden Institute of Advanced Computer Science (LIACS). https://openaccess.leidenuniv.nl/handle/1887/67292

[29]

Adalberto L. Simeone, Eduardo Velloso, and Hans Gellersen. 2015. Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 3307–3316. https://doi.org/10.1145/2702123.2702389

Digital Library

[30]

Stereolabs. 2020. A Next-generation Stereo Camera. https://www.stereolabs.com/zed-2/

[31]

Imangi Studios. 2021. Temple Run Mobile Game. https://play.google.com/store/apps/details?id=com.imangi.templerun

[32]

Jason Thompson, Mark Stevenson, Jasper S Wijnands, Kerry A Nice, Gideon DPA Aschwanden, Jeremy Silver, Mark Nieuwenhuijsen, Peter Rayner, Robyn Schofield, Rohit Hariharan, and Christopher N Morrison. 2020. A global analysis of urban design types and road transport injury: an image processing study. Lancet Planet Health 4(2020), 32–42. https://doi.org/10.1016/S2542-5196(19)30263-3

[33]

M. Tonnis, C. Sandor, G. Klinker, C. Lange, and H. Bubb. 2005. Experimental evaluation of an augmented reality visualization for directing a car driver’s attention. Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05) (2005). https://doi.org/10.1109/ismar.2005.31

[34]

Emma van Amersfoorth, Lotte Roefs, Quinta Bonekamp, Laurent Schuermans, and Bastian Pfleging. 2019. Increasing driver awareness through translucency on windshield displays. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings (2019). https://doi.org/10.1145/3349263.3351911

Digital Library

[35]

VideoGameGeek. 2021. Video Game Guide to Genres. https://videogamegeek.com/wiki/page/Video_Game_Guide_to_Genres

[36]

G. Stewart Von Itzstein, Mark Billinghurst, Ross T. Smith, and Bruce H. Thomas. 2017. Augmented Reality Entertainment: Taking Gaming Out of the Box. Springer International Publishing, Cham, 1–9. https://doi.org/10.1007/978-3-319-08234-9_81-1

[37]

WayRay. 2021. Holographic Augmented Reality Display. https://wayray.com/

[38]

Richard Wetzel, Rod McCall, Anne-Kathrin Braun, and Wolfgang Broll. 2008. Guidelines for designing augmented reality games. Proceedings of the 2008 Conference on Future Play Research, Play, Share - Future Play ’08 (2008). https://doi.org/10.1145/1496984.1497013

Digital Library

[39]

Gesa Wiegand, Christian Mai, Kai Holländer, and Heinrich Hussmann. 2019. InCarAR: A Design Space Towards 3D Augmented Reality Applications in Vehicles. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI ’19 (2019). https://doi.org/10.1145/3342197.3344539

Digital Library

[40]

Julie R. Williamson, Mark McGill, and Khari Outram. 2019. PlaneVR: Social Acceptability of Virtual Reality for Aeroplane Passengers. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300310

Digital Library

[41]

Zhengxia Zou. 2020. Castle in the Sky: Dynamic Sky Replacement and Harmonization in Videos. arxiv:2010.11800 [cs.CV]

Cited By

Stampf AColley M(2024)Deriving Non-Driving-Related Activities in Highly Automated Driving via an Autoethnographic Approach by Traveling Canada in a Recreational VehicleProceedings of Mensch und Computer 202410.1145/3670653.3670663(279-287)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3670663
Colley MSpeidel OStrohbeck JRixen JBelz JRukzio E(2024)Effects of Uncertain Trajectory Prediction Visualization in Highly Automated Vehicles on Trust, Situation Awareness, and Cognitive LoadProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314087:4(1-23)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631408
Deng LLi DChen J(2023)Compatibility as a Prerequisite: Research on the Factors Influencing the Continuous Use Intention of In-vehicle Games Based on Diffusion of Innovations TheorySAGE Open10.1177/2158244023121790913:4Online publication date: 19-Dec-2023
https://doi.org/10.1177/21582440231217909
Show More Cited By

Index Terms

In-cAR Gaming: Exploring the use of AR headsets to Leverage Passenger Travel Environments for Mixed Reality Gameplay

Index terms have been assigned to the content through auto-classification.

Recommendations

PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR Experiences
UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

We present PassengXR, an open-source toolkit for creating passenger eXtended Reality (XR) experiences in Unity. XR allows travellers to move beyond the physical limitations of in-vehicle displays, rendering immersive virtual content based on - or ...
I Am The Passenger: How Visual Motion Cues Can Influence Sickness For In-Car VR
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

This paper explores the use of VR Head Mounted Displays (HMDs) in-car and in-motion for the first time. Immersive HMDs are becoming everyday consumer items and, as they offer new possibilities for entertainment and productivity, people will want to use ...
Challenges in passenger use of mixed reality headsets in cars and other transportation
Abstract
This paper examines key challenges in supporting passenger use of augmented and virtual reality headsets in transit. These headsets will allow passengers to break free from the restraints of physical displays placed in constrained environments ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

3066 pages

ISBN:9781450391566

DOI:10.1145/3491101

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster
Research
Refereed limited

Funding Sources

European Research Council

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
566
Total Downloads

Downloads (Last 12 months)214
Downloads (Last 6 weeks)18

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Stampf AColley M(2024)Deriving Non-Driving-Related Activities in Highly Automated Driving via an Autoethnographic Approach by Traveling Canada in a Recreational VehicleProceedings of Mensch und Computer 202410.1145/3670653.3670663(279-287)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3670663
Colley MSpeidel OStrohbeck JRixen JBelz JRukzio E(2024)Effects of Uncertain Trajectory Prediction Visualization in Highly Automated Vehicles on Trust, Situation Awareness, and Cognitive LoadProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314087:4(1-23)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631408
Deng LLi DChen J(2023)Compatibility as a Prerequisite: Research on the Factors Influencing the Continuous Use Intention of In-vehicle Games Based on Diffusion of Innovations TheorySAGE Open10.1177/2158244023121790913:4Online publication date: 19-Dec-2023
https://doi.org/10.1177/21582440231217909
Medeiros DDubus RWilliamson JWilson GPöhlmann KMcGill M(2023)Surveying the Social Comfort of Body, Device, and Environment-Based Augmented Reality Interactions in Confined Passenger Spaces Using Mixed Reality Composite VideosProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109237:3(1-25)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610923
Wilson GMcGill MMedeiros DBrewster S(2023)A Lack of Restraint: Comparing Virtual Reality Interaction Techniques for Constrained Transport SeatingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324708429:5(2390-2400)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3247084
Berfin Ince ECha KCho J(2023)An investigation into generation Z’s mindsets of entertainment in an autonomous vehicleEntertainment Computing10.1016/j.entcom.2023.10055046(100550)Online publication date: May-2023
https://doi.org/10.1016/j.entcom.2023.100550
McGill MWilson GMedeiros DBrewster S(2022)PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR ExperiencesProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545657(1-15)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545657

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents