Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Factors Influencing Engagement in Hybrid Virtual and Augmented Reality

Published: 11 September 2023 Publication History

Abstract

Hybridity in immersive technologies has not been studied for factors that are likely to influence engagement. A noticeable factor is the spatial enclosure that defines where users meet. This involves a mutual object of interest, contents that the users may generate around the object, and the proximity between users. This study examines these factors, namely how object interactivity, user-generated contents (UGC) and avatar proximity influence engagement. We designed a Hybrid Virtual and Augmented Reality (HVAR) environment that supports paired users to experience cultural heritage in both Virtual Reality (VR) and Augmented Reality (AR). A user study was conducted with 60 participants, providing assessments of engagement and presence via questionnaires, together with mobile electroencephalogram (mEEG) and user activity data that measures VR user engagement in real-time. Our findings provide insights into how engagement between users can occur in HVAR environments for the future hybrid reality with multi-device connectivity.

References

[1]
Rebecca L. Acabchuk, Mareyna A. Simon, Spencer Low, Julie M. Brisson, and Blair T. Johnson. 2021. Measuring meditation progress with a consumer-grade EEG device: Caution from a randomized controlled trial. Mindfulness 12, 1 (2021), 68–81.
[2]
Mafkereseb Kassahun Bekele, Roberto Pierdicca, Emanuele Frontoni, and E. V. A. Savina Malinverni. 2018. A survey of augmented, virtual, and mixed reality for cultural heritage. ACM Journal on Computing and Cultural Heritage 11, 2 (2018), 36.
[3]
Chris Berka, Daniel J. Levendowski, Michelle N. Lumicao, Alan Yau, Gene Davis, Vladimir T. Zivkovic, Richard E. Olmstead, Patrice D. Tremoulet, and Patrick L. Craven. 2007. EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviation Space and Environmental Medicine 78, 5 II (2007).
[4]
Nadia Bianchi-Berthouze. 2013. Understanding the role of body movement in player engagement. Human-Computer Interaction 28, 1 (2013), 40–75.
[5]
Benjamin Blankertz, Michael Tangermann, Carmen Vidaurre, Siamac Fazli, Claudia Sannelli, Stefan Haufe, Cecilia Maeder, Lenny Ramsey, Irene Sturm, Gabriel Curio, and Klaus Robert Müller. 2010. The Berlin brain-computer interface: Non-medical uses of BCI technology.
[6]
Doug A. Bowman and Ryan P. McMahan. 2007. Virtual reality: How much immersion is enough? Computer 40, 7 (Jul.2007), 36–43.
[7]
Jennifer Brade, Mario Lorenz, Marc Busch, Niels Hammer, Manfred Tscheligi, and Philipp Klimant. 2017. Being there again - presence in real and virtual environments and its relation to usability and user experience using a mobile navigation task. International Journal of Human-Computer Studies 101, March 2016 (2017), 76–87.
[8]
Jeanne H. Brockmyer, Christine M. Fox, Kathleen A. Curtiss, Evan McBroom, Kimberly M. Burkhart, and Jacquelyn N. Pidruzny. 2009. The development of the game engagement questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social Psychology 45, 4 (Jul.2009), 624–634.
[9]
Emily Brown and Paul Cairns. 2004. A grounded investigation of game immersion. In Conference on Human Factors in Computing Systems - Proceedings.
[10]
Marc Busch, Mario Lorenz, Manfred Tscheligi, Christina Hochleitner, and Trenton Schulz. 2014. Being there for real: Presence in real and virtual environments and its relation to usability. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI’14 Helsinki, Finland), Association for Computing Machinery, New York, NY, 117–126.
[11]
Cédric Cannard, Helané Wahbeh, and Arnaud Delorme. 2021. Electroencephalography correlates of well-being using a low-cost wearable system. Frontiers in Human Neuroscience 15, December (2021).
[12]
Eugene Ch’ng and Neil Cooke. 2015. User study on 3D multitouch interaction (3DMi) and gaze on surface computing. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9187 (2015), 425–433.
[13]
Eugene Ch’ng, Yue Li, Shengdan Cai, and Fui Theng Leow. 2020. The effects of VR environments on the acceptance, experience, and expectations of cultural heritage learning. Journal on Computing and Cultural Heritage 13, 1 (2020), 1–20.
[14]
James J. Cummings and Jeremy N. Bailenson. 2016. How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology 19, 2 (2016), 272–309.
[15]
Marcel Danesi. 2016. The Semiotics of Emoji: The Rise of Visual Language in the Age of the Internet. Bloomsbury Publishing.
[16]
Kevin Doherty and Gavin Doherty. 2018. Engagement in HCI: Conception, theory and measurement. ACM Computing Surveys (CSUR) 51, 5 (2018), 99:1—-99:39.
[17]
Steven Dow. 2007. User engagement in physically embodied narrative experiences. Creativity and Cognition 2007, CC2007 - Seeding Creativity: Tools, Media, and Environments (2007), 280.
[18]
John Howard Falk and Lynn Diane Dierking. 2000. Learning from Museums: Visitor Experiences and the Making of Meaning. arxiv:arXiv:1011.1669v3
[19]
Frederick G. Freeman, Peter J. Mikulka, Lawrence J. Prinzel, and Mark W. Scerbo. 1999. Evaluation of an adaptive automation system using three EEG indices with a visual tracking task. Biological Psychology 50, 1 (1999), 61–76.
[20]
Lukas Gehrke, Sezen Akman, Pedro Lopes, Albert Chen, Avinash Kumar Singh, Hsiang-ting Chen, Chin-Teng Lin, and Klaus Gramann. 2019. Detecting visuo-haptic mismatches in virtual reality using the prediction error negativity of event-related brain potentials. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–11.
[21]
Philippe Gelisse, Pierre Genton, and Arielle Crespel. 2020. Failure to recognize muscular artifacts on the EEG may cause a wrong diagnosis of myoclonic status epilepticus. Epilepsy and Behavior Reports 14 (2020), 0–2.
[22]
Sergio Giraldo and Rafael Ramirez. 2013. Brain-activity-driven real-time music emotive control. 3rd International Conference on Music & EmotionJune (2013), 11–15.
[23]
Edward T. Hall. 1973. The hidden dimension. Leonardo (1973).
[24]
Mariam Hassib, Stefan Schneegass, Philipp Eiglsperger, Niels Henze, Albrecht Schmidt, and Florian Alt. 2017. EngageMeter: A system for implicit audience engagement sensing using electroencephalography. Conference on Human Factors in Computing Systems - Proceedings 2017-May (2017), 5114–5119.
[25]
Carrie Heeter. 1992. Being there: The subjective experience of presence. Presence: Teleoperators and Virtual Environments 1, 2 (Jan.1992), 262–271.
[26]
Richard M. Held and Nathaniel I. Durlach. 1992. Telepresence. Presence: Teleoperators and Virtual Environments 1, 1 (Jan.1992), 109–112.
[27]
Yvonne Hellin-Hobbs. 2010. The constructivist museum and the web.
[28]
Patrick Hennig, Philipp Berger, Christoph Meinel, Maria Graber, Jens Hildebrandt, Stefan Lehmann, and Cathleen Ramson. 2013. Tracking visitor engagement in the blogosphere for leveraging rankings. In Proceedings - SocialCom/PASSAT/BigData/EconCom/BioMedCom 2013.
[29]
Wijnand Ijsselsteijn and Giuseppe Riva. 2003. Being there: The experience of presence in mediated environments. Being There: Concepts, Effects and Measurement of user Presence in Synthetic Environments (2003), 14. https://doi.org/citeulike-article-id:4444927
[30]
Alejandro Jaimes, Mounia Lalmas, and Yana Volkovich. 2011. First international workshop on social media engagement (SoME 2011). In Proceedings of the 20th International Conference Companion on World Wide Web, WWW 2011.
[31]
Maria Koutamanis, Helen G. M. Vossen, and Patti M. Valkenburg. 2015. Adolescents’ comments in social media: Why do adolescents receive negative feedback and who is most at risk? Computers in Human Behavior 53 (2015), 486–494.
[32]
Olave E. Krigolson, Mathew R. Hammerstrom, Wande Abimbola, Robert Trska, Bruce W. Wright, Kent G. Hecker, and Gordon Binsted. 2021. Using Muse: Rapid mobile assessment of brain performance. Frontiers in Neuroscience 15, January (2021), 1–11.
[33]
Olave E. Krigolson, Chad C. Williams, and Francisco L. Colino. 2017. Using portable EEG to assess human visual attention. Vol. 10284. 56–65.
[34]
Cliff Lampe. 2013. Behavioral trace data for analyzing online communities. In The SAGE Handbook of Digital Technology Research.
[35]
Jane Lessiter, Jonathan Freeman, Edmund Keogh, and Jules Davidoff. 2001. A cross-media presence questionnaire: The ITC-sense of presence inventory. Presence 10, 3 (2001), 282–297.
[36]
Yue Li, Eugene Ch’ng, Shengdan Cai, and Simon See. 2018. Multiuser interaction with hybrid VR and AR for cultural heritage objects. In Digital Heritage 2018. IEEE, San Francisco, CA, USA.
[37]
Yue Li, Eugene Ch’ng, Sue Cobb, and Simon See. 2022. Presence and communication in hybrid virtual and augmented reality environments. PRESENCE: Virtual and Augmented Reality (2022).
[38]
Yue Li, Paul Tennent, and Sue Cobb. 2019. Appropriate control methods for mobile virtual exhibitions. In VRTCH’18. Springer, Brasov, Romania.
[39]
Matthew Lombard and Theresa Ditton. 1997. At the heart of it all: The concept of presence. Journal of Computer-Mediated Communication 3, 2 (1997), 0.
[40]
Matthew Lombard, Theresa B. Ditton, and Lisa Weinstein. 2009. Measuring presence: The temple presence inventory. In Proceedings of the 12th Annual International Workshop on Presence. 1–15. http://www.temple.edu/ispr/prev_conferences/proceedings/2009/Lombard_et_al.pdf.
[41]
Jack M. Loomis. 1992. Distal attribution and presence. Presence: Teleoperators and Virtual Environments - Premier issue 1, 1 (1992), 113–119.
[42]
Nicolai Marquardt and Saul Greenberg. 2012. Informing the design of proxemic interactions. IEEE Pervasive Computing (2012).
[43]
Alison McMahan. 2003. Immersion, engagement, and presence: A method for analyzing 3-D video games. In The Video Game Theory Reader.
[44]
Ryan P. McMahan, Doug A. Bowman, David J. Zielinski, and Rachael B. Brady. 2012. Evaluating display fidelity and interaction fidelity in a virtual reality game. IEEE Transactions on Visualization and Computer Graphics 18, 4 (2012), 626–633.
[45]
Timothy McMahan, Ian Parberry, and Thomas D. Parsons. 2015. Evaluating player task engagement and arousal using electroencephalography. Procedia Manufacturing 3, Ahfe (2015), 2303–2310.
[46]
Michael Meehan, Brent Insko, Mary Whitton, and Frederick P. Brooks. 2002. Physiological measures of presence in stressful virtual environments. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH’02. ACM Press, New York, New York, USA, 645.
[47]
Marvin Minsky. 1980. Telepresence. OMNI MagazineJune (1980), 1–6.
[48]
Diego Monteiro, Hai Ning Liang, Andrew Abel, Nilufar Bahaei, and Rita De Cassia Monteiro. 2018. Evaluating engagement of virtual reality games based on first and third person perspective using EEG and subjective metrics. Proceedings - 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2018 (2018), 53–60.
[49]
Helen Neale and Sarah Nichols. 2001. Theme-based content analysis: A flexible method for virtual environment evaluation. International Journal of Human Computer Studies 55, 2 (2001), 167–189.
[50]
Heather L. O’Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human Computer Studies 112, December 2017 (2018), 28–39.
[51]
Heather L. O’Brien and Elaine G. Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology 59, 6 (2008), 938–955.
[52]
Heather L. O’Brien and Elaine G. Toms. 2010. The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology 61, 1 (Jan.2010), 50–69.
[53]
Jeeyun Oh and S. Shyam Sundar. 2016. User engagement with interactive media: A communication perspective. In Why Engagement Matters. Springer International Publishing, Cham, 177–198.
[54]
Jorge Peña and Seung-Chul Yoo. 2014. Under pressure: Avatar appearance and cognitive load effects on attitudes, trustworthiness, bidding, and interpersonal distance in a virtual store. Presence: Teleoperators and Virtual Environments 23, 1 (Feb.2014), 18–32.
[55]
Matthew Pike and Eugene Ch’ng. 2016. Evaluating virtual reality experience and performance: A brain based approach. In Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry - VRCAI’16. ACM Press, New York, New York, USA, 469–474.
[56]
Alan T. Pope, Edward H. Bogart, and Debbie S. Bartolome. 1995. Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology 40, 1–2 (1995), 187–195.
[57]
Lawrence J. Prinzel, Frederick G. Freeman, Mark W. Scerbo, Peter J. Mikulka, and Alan T. Pope. 2000. A closed-loop system for examining psychophysiological measures for adaptive task allocation. International Journal of Aviation Psychology 10, 4 (2000), 393–410.
[58]
Holger Regenbrecht. 2002. Measuring presence in augmented reality environments: Design and a first test of a questionnaire. Annual International Workshop Presence (2002), 1–4. https://www.google.com/%5Cnpapers3://publication/uuid/BE89B550-B84B-4411-9464-71837C29609F.
[59]
Marco C. Rozendaal, David V. Keyson, Huib de Ridder, and Peter O. Craig. 2009. Game feature and expertise effects on experienced richness, control and engagement in game play. AI and Society (2009).
[60]
Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The experience of presence: Factor analytic insights. Presence: Teleoperators and Virtual Environments 10, 3 (2001), 266–281.
[61]
Martijn J. Schuemie, Peter van der Straaten, Merel Krijn, and Charles A. P. G. van der Mast. 2001. Research on presence in virtual reality: A survey. CyberPsychology & Behavior 4, 2 (2001), 183–201.
[62]
Neta Shaby, Orit Ben Zvi Assaraf, and Tali Tal. 2017. The particular aspects of science museum exhibits that encourage students’ engagement. Journal of Science Education and Technology (2017).
[63]
Thomas B. Sheridan. 1992. Musings on telepresence and virtual presence. Presence: Teleoperators & Virtual Environments 1, 1 (1992), 120–126.
[64]
Richard Skarbez, Frederick P. Brooks, and Mary C. Whitton. 2017. A survey of presence and related concepts. ACM Comput. Surv. Article 50, 96 (2017).
[65]
Mel Slater. 2009. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3549–3557.
[66]
Mel Slater, Christoph Guger, Guenter Edlinger, Robert Leeb, Gert Pfurtscheller, Angus Antley, Maia Garau, Andrea Brogni, and Doron Friedman. 2006. Analysis of physiological responses to a social situation in an immersive virtual environment. Presence: Teleoperators and Virtual Environments 15, 5 (2006), 553–569.
[67]
Mel Slater, Martin Usoh, and Anthony Steed. 1994. Depth of presence in virtual environments. Presence: Teleoperators and Virtual Environments 3 (1994), 130–144.
[68]
Mel Slater and Sylvia Wilbur. 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators and Virtual Environments 6, 6 (Dec.1997), 603–616.
[69]
Jonathan Steuer. 1992. Defining virtual reality: Dimensions determining telepresence. Journal of Communication 42, 4 (1992), 73–93.
[70]
Stella Sylaiou, Katerina Mania, Athanasis Karoulis, and Martin White. 2010. Exploring the relationship between presence and enjoyment in a virtual museum. International Journal of Human Computer Studies 68, 5 (2010), 243–253.
[71]
Arthur Tang, Frank Biocca, and Lynette Lim. 2004. Comparing differences in presence during social interaction in augmented reality versus virtual reality environments : An exploratory study. 7th Annual International Workshop on Presence (2004), 204–208. papers3://publication/uuid/81D1A395-AA28-4DC5-9C50-3FD2A207F6B6
[72]
Paul Tennent, Sarah Martindale, Steve Benford, and Dimitrios Darzentas. 2020. Thresholds : Embedding virtual reality in the museum. Journal on Computing and Cultural Heritage 13, 2 (2020), 12:1–12:35.
[73]
Daniel Vogel and Ravin Balakrishnan. 2004. Interactive public ambient displays: Transitioning from implicit to explicit, public to personal, interaction with multiple users. UIST: Proceedings of the Annual ACM Symposium on User Interface Software and Technology 6, 2 (2004), 137–146.
[74]
Cassandra M. Wilkinson, Jennifer I. Burrell, Jonathan W. P. Kuziek, Sibi Thirunavukkarasu, Brian H. Buck, and Kyle E. Mathewson. 2020. Predicting stroke severity with a 3-min recording from the Muse portable EEG system for rapid diagnosis of stroke. Scientific Reports 10, 1 (2020), 1–11.
[75]
Bob G. Witmer, Christian J. Jerome, and Michael J. Singer. 2005. The factor structure of the presence questionnaire. Presence: Teleoperators and Virtual Environments 14, 3 (2005), 298–312.
[76]
Bob G. Witmer and Michael J. Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments (Aug.1998), 225–240. http://proceedings.spiedigitallibrary.org/proceeding.aspx?doi=10.1117/12.2233447.

Cited By

View all
  • (2024)Designing Effective VR Learning EnvironmentsCreating Immersive Learning Experiences Through Virtual Reality (VR)10.4018/979-8-3693-6407-9.ch004(77-104)Online publication date: 22-Nov-2024
  • (2024)Exploring the Kuroko Paradigm: The Effect of Enhancing Virtual Humans with Reality Actuators in Augmented RealityProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652945(79-90)Online publication date: 4-Apr-2024
  • (2024)Meaning Follows Purpose: Unravelling the Architectural Design Conventions in the Contemporary MetaverseProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642519(1-22)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction
ACM Transactions on Computer-Human Interaction  Volume 30, Issue 4
August 2023
614 pages
ISSN:1073-0516
EISSN:1557-7325
DOI:10.1145/3623486
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2023
Online AM: 30 March 2023
Accepted: 21 February 2023
Revised: 13 February 2023
Received: 26 August 2021
Published in TOCHI Volume 30, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Virtual reality
  2. augmented reality
  3. mixed reality
  4. engagement
  5. presence
  6. electroencephalogram
  7. mEEG

Qualifiers

  • Research-article

Funding Sources

  • National Natural Science Foundation of China
  • Natural Science Foundation of the Jiangsu Higher Education Institutions of China
  • Xi’an Jiaotong-Liverpool University

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,004
  • Downloads (Last 6 weeks)85
Reflects downloads up to 22 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Designing Effective VR Learning EnvironmentsCreating Immersive Learning Experiences Through Virtual Reality (VR)10.4018/979-8-3693-6407-9.ch004(77-104)Online publication date: 22-Nov-2024
  • (2024)Exploring the Kuroko Paradigm: The Effect of Enhancing Virtual Humans with Reality Actuators in Augmented RealityProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652945(79-90)Online publication date: 4-Apr-2024
  • (2024)Meaning Follows Purpose: Unravelling the Architectural Design Conventions in the Contemporary MetaverseProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642519(1-22)Online publication date: 11-May-2024
  • (2023)MetaverseMetaverse Applications for Intelligent Healthcare10.4018/978-1-6684-9823-1.ch003(93-158)Online publication date: 10-Nov-2023
  • (2023)Comparative Analysis of Artefact Interaction and Manipulation Techniques in VR Museums: A Study of Performance and User Experience2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00091(761-770)Online publication date: 16-Oct-2023
  • (2023)Early Aesthetics Integration in Digital Twin Dashboards in Isolated Agriculture: Data Modelling and Remote Networks2023 IEEE Asia-Pacific Conference on Geoscience, Electronics and Remote Sensing Technology (AGERS)10.1109/AGERS61027.2023.10490774(77-83)Online publication date: 19-Dec-2023

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media