VR PreM+: An Immersive Pre-learning Branching Visualization System for Museum Tours
Authors: 
Ze Gao, 
Xiang Li, 
Changkun Liu, Xian Wang, Anqi Wang, Liang Yang, Yuyang Wang, Pan Hui, Tristan BraudAuthors Info & Claims
Ze Gao, Xiang Li, Changkun Liu, Xian Wang, Anqi Wang, Liang Yang, Yuyang Wang, Pan Hui, Tristan BraudAuthors Info & ClaimsPages 374 - 385
Abstract
We present VR PreM+, an innovative VR system designed to enhance web exploration beyond traditional computer screens. Unlike static 2D displays, VR PreM+ leverages 3D environments to create an immersive pre-learning experience. Using keyword-based information retrieval allows users to manage and connect various content sources in a dynamic 3D space, improving communication and data comparison. We conducted preliminary and user studies that demonstrated efficient information retrieval, increased user engagement, and a greater sense of presence. These findings yielded three design guidelines for future VR information systems: display, interaction, and user-centric design. VR PreM+ bridges the gap between traditional web browsing and immersive VR, offering an interactive and comprehensive approach to information acquisition. It holds promise for research, education, and beyond.
Supplementary Material
System Demonstration Video
- Download
- 81.48 MB
References
[1]
Matthew Addis, Mike Boniface, Simon Goodall, Paul Grimwood, Sanghee Kim, Paul Lewis, Kirk Martinez, and Alison Stevenson. 2003. SCULPTEUR: Towards a new paradigm for multimedia museum information handling. In International Semantic Web Conference. Springer, 582–596.
[2]
Mohamed Ally. 2004. Foundations of educational theory for online learning. Theory and practice of online learning 2 (2004), 15–44.
[3]
Ravin Balakrishnan. 2006. Why Aren’t We Using 3D User Interfaces, and Will We Ever?. In 3D User Interfaces (3DUI’06). IEEE Computer Society, 141–141.
[4]
Soumya C Barathi, Daniel J Finnegan, Matthew Farrow, Alexander Whaley, Pippa Heath, Jude Buckley, Peter W Dowrick, Burkhard C Wuensche, James LJ Bilzon, Eamonn O’Neill, 2018. Interactive feedforward for improving performance and maintaining intrinsic motivation in VR exergaming. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–14.
[5]
Jim Blackaby and Beth Sandore. 1997. Building integrated museum information retrieval systems: Practical approaches to data organization and access. Archives and museum informatics 11, 2 (1997), 117–146.
[6]
Jelte E Bos. 2015. Less sickness with more motion and/or mental distraction. Journal of Vestibular Research 25, 1 (2015), 23–33.
[7]
Ian Brown. 1999. Developing a virtual reality user interface (VRUI) for geographic information retrieval on the Internet. Transactions in GIS 3, 3 (1999), 207–220.
[8]
Ricardo Campos, Gaël Dias, Alípio M Jorge, and Adam Jatowt. 2014. Survey of temporal information retrieval and related applications. ACM Computing Surveys (CSUR) 47, 2 (2014), 1–41.
[9]
Carolina Cruz-Neira, Daniel J Sandin, and Thomas A DeFanti. 1993. Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 135–142.
[10]
Susan Dumais, Edward Cutrell, Jonathan J Cadiz, Gavin Jancke, Raman Sarin, and Daniel C Robbins. 2003. Stuff I’ve seen: a system for personal information retrieval and re-use. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. 72–79.
[11]
Shulin Fang. 2020. Visualization of information retrieval in smart library based on virtual reality technology. Complexity 2020 (2020).
[12]
Ze Gao. 2023. Envisioning an Immersive Multi-Screen VR System for Museum Archive Browsing. In Proceedings of EVA London 2023. BCS Learning & Development, 160–162.
[13]
John F Golding. 2006. Predicting individual differences in motion sickness susceptibility by questionnaire. Personality and Individual differences 41, 2 (2006), 237–248.
[14]
Maria Harrington. 2020. Augmented and Virtual Reality Application Design for Immersive Learning Research Using Virtual Nature: Making Knowledge Beautiful and Accessible with Information Fidelity. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Talks. 1–2.
[15]
WC Harris, PA Hancock, and EJ Arthur. 1993. The effect of taskload projection on automation use, performance, and workload. The Adaptive Function allocation for Intelligent Cockpits (AFAIQ program: Interim research and guidelines for the application of adaptive automation. (1993), 25.
[16]
Jianfei Hu, Bin Zhang, and Di Wang. 2019. Application of Virtual Reality Technology in Library Visual Information Retrieval. In IOP Conference Series: Materials Science and Engineering, Vol. 569. IOP Publishing, 032062.
[17]
Hanna Kallio, Anna-Maija Pietilä, Martin Johnson, and Mari Kangasniemi. 2016. Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. Journal of advanced nursing 72, 12 (2016), 2954–2965.
[18]
Marijn Koolen, Jaap Kamps, and Vincent de Keijzer. 2009. Information retrieval in cultural heritage. Interdisciplinary Science Reviews 34, 2-3 (2009), 268–284.
[19]
Kit Yung Lam, Lik Hang Lee, Tristan Braud, and Pan Hui. 2019. M2a: A framework for visualizing information from mobile web to mobile augmented reality. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. IEEE, 1–10.
[20]
Kit Yung Lam, Lik Hang Lee, and Pan Hui. 2021. A2w: Context-aware recommendation system for mobile augmented reality web browser. In Proceedings of the 29th ACM International Conference on Multimedia. 2447–2455.
[21]
Lik Hang Lee, Tristan Braud, Kit Yung Lam, Yui Pan Yau, and Pan Hui. 2020. From seen to unseen: Designing keyboard-less interfaces for text entry on the constrained screen real estate of Augmented Reality headsets. Pervasive and Mobile Computing 64 (2020), 101148.
[22]
Lu Li and Ji Zhou. 2016. Virtual reality technology based developmental designs of multiplayer-interaction-supporting exhibits of science museums: taking the exhibit of" virtual experience on an aircraft carrier" in China science and technology museum as an example. In Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1. 409–412.
[23]
Xinying Li. 2018. Research on application of virtual reality technology in information retrieval. In IOP Conference Series: Materials Science and Engineering, Vol. 423. IOP Publishing, 012098.
[24]
Xiang Li, Yuzheng Chen, Rakesh Patibanda, and Florian ’Floyd’ Mueller. 2021. VrCAPTCHA: Exploring CAPTCHA Designs in Virtual Reality. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 167, 4 pages. https://doi.org/10.1145/3411763.3451985
[25]
Xiang Li, Xiaohang Tang, Xin Tong, Rakesh Patibanda, Florian ’Floyd’ Mueller, and Hai-Ning Liang. 2021. Myopic Bike and Say Hi: Games for Empathizing with The Myopic. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play. 333–338.
[26]
Xiang Li, Jin-Du Wang, John J. Dudley, and Per Ola Kristensson. 2023. Swarm Manipulation in Virtual Reality. In The 2023 ACM Symposium on Spatial User Interaction (SUI ’23) (Sydney, NSW, Australia). ACM, New York, NY, USA, 11 Pages. https://doi.org/10.1145/3607822.3614519
[27]
Gi-zen Liu, Tzu-chien Liu, Chih-chung Lin, Yu-ling Kuo, and Gwo-jen Hwang. 2016. Identifying learning features and models for context-aware ubiquitous learning with phenomenological research method. International Journal of Mobile Learning and Organisation 10, 4 (2016), 238–262.
[28]
Alan MacLennan. 2007. Design of virtual worlds for accessing information: discovery of user preferences.Ph. D. Dissertation.
[29]
Ann McNamara, Katherine Boyd, David Oh, Ryan Sharpe, and Annie Suther. 2018. Using eye tracking to improve information retrieval in virtual reality. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 242–243.
[30]
Yoichi Motomura, Kaori Yoshida, and Kazunori Fujimoto. 2000. Generative user models for adaptive information retrieval. In Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics.’cybernetics evolving to systems, humans, organizations, and their complex interactions’(cat. no. 0, Vol. 1. IEEE, 665–670.
[31]
Florian ’Floyd’ Mueller, Rakesh Patibanda, Richard Byrne, Zhuying Li, Yan Wang, Josh Andres, Xiang Li, Jonathan Marquez, Stefan Greuter, Jonathan Duckworth, 2021. Limited control over the body as intriguing play design resource. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.
[32]
Florian ’Floyd’ Mueller, Nathan Semertzidis, Josh Andres, Joe Marshall, Steve Benford, Xiang Li, Louise Matjeka, and Yash Mehta. 2023. Towards Understanding the Design of Intertwined Human-Computer Integrations. ACM Trans. Comput.-Hum. Interact. (apr 2023). https://doi.org/10.1145/3590766 Just Accepted.
[33]
Dan Olsen. 2008. Interactive viscosity. In Proceedings of the 21st annual ACM symposium on User interface software and technology. 1–2.
[34]
Michael Otto, Eva Lampen, Philipp Agethen, Mareike Langohr, Gabriel Zachmann, and Enrico Rukzio. 2019. A virtual reality assembly assessment benchmark for measuring VR performance & limitations. Procedia CIRP 81 (2019), 785–790.
[35]
Zhigeng Pan, Adrian David Cheok, Hongwei Yang, Jiejie Zhu, and Jiaoying Shi. 2006. Virtual reality and mixed reality for virtual learning environments. Computers & graphics 30, 1 (2006), 20–28.
[36]
Rakesh Patibanda, Chris Hill, Aryan Saini, Xiang Li, Yuzheng Chen, Andrii Matviienko, Jarrod Knibbe, Elise van den Hoven, and Florian ‘Floyd’ Mueller. 2023. Auto-Paizo Games: Towards Understanding the Design of Games That Aim to Unify a Player’s Physical Body and the Virtual World. Proc. ACM Hum.-Comput. Interact. 7, CHI PLAY, Article 408 (oct 2023), 26 pages. https://doi.org/10.1145/3611054
[37]
Rakesh Patibanda, Aryan Saini, Nathalie Overdevest, Maria F. Montoya, Xiang Li, Yuzheng Chen, Shreyas Nisal, Josh Andres, Jarrod Knibbe, Elise van den Hoven, and Florian ‘Floyd’ Mueller. 2023. Fused Spectatorship: Designing Bodily Experiences Where Spectators Become Players. Proc. ACM Hum.-Comput. Interact. 7, CHI PLAY, Article 403 (oct 2023), 34 pages. https://doi.org/10.1145/3611049
[38]
Mykola Pechenizkiy and Toon Calders. 2007. A framework for guiding the museum tours personalization. In Proceedings of the workshop on personalised access to cultural heritage (patch07). 11–28.
[39]
David A Plecher, Maximilian Wandinger, and Gudrun Klinker. 2019. Mixed reality for cultural heritage. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 1618–1622.
[40]
Sumitra Pokhrel and Roshan Chhetri. 2021. A literature review on impact of COVID-19 pandemic on teaching and learning. Higher Education for the Future 8, 1 (2021), 133–141.
[41]
Anna Puig, Inmaculada Rodríguez, Josep Ll Arcos, Juan A Rodríguez-Aguilar, Sergi Cebrián, Anton Bogdanovych, Núria Morera, Antoni Palomo, and Raquel Piqué. 2020. Lessons learned from supplementing archaeological museum exhibitions with virtual reality. Virtual Reality 24, 2 (2020), 343–358.
[42]
Jaziar Radianti, Tim A Majchrzak, Jennifer Fromm, and Isabell Wohlgenannt. 2020. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education 147 (2020), 103778.
[43]
Jonathan C Roberts, Joseph W Mearman, Peter WS Butcher, Hayder Mahdi Al-Maneea, and Panagiotis D Ritsos. 2021. 3D Visualisations Should Not be Displayed Alone-Encouraging a Need for Multivocality in Visualisation. arXiv preprint arXiv:2108.04680 (2021).
[44]
Lisa C Roberts. 2014. From knowledge to narrative: Educators and the changing museum. Smithsonian Institution.
[45]
Irene Rubino, Claudia Barberis, Jetmir Xhembulla, and Giovanni Malnati. 2015. Integrating a location-based mobile game in the museum visit: Evaluating visitors’ behaviour and learning. Journal on Computing and Cultural Heritage (JOCCH) 8, 3 (2015), 1–18.
[46]
Guy Schofield, Gareth Beale, Nicole Beale, Martin Fell, Dawn Hadley, Jonathan Hook, Damian Murphy, Julian Richards, and Lewis Thresh. 2018. Viking VR: designing a virtual reality experience for a museum. In Proceedings of the 2018 Designing Interactive Systems Conference. 805–815.
[47]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence, 4 (6), 103-108. (2017).
[48]
T Schubert, F Friedmann, and H Regenbrecht. 2001. Igroup presence questionnaire. Teleoperators Virtual Environ 41 (2001), 115–124.
[49]
Alistair G Sutcliffe, Charalambos Poullis, Andreas Gregoriades, Irene Katsouri, Aimilia Tzanavari, and Kyriakos Herakleous. 2019. Reflecting on the design process for virtual reality applications. International Journal of Human–Computer Interaction 35, 2 (2019), 168–179.
[50]
Radu-Daniel Vatavu and Matei Mancas. 2014. Visual attention measures for multi-screen TV. In Proceedings of the ACM international conference on interactive experiences for TV and online video. 111–118.
[51]
Radu-Daniel Vatavu and Matei Mancas. 2015. Evaluating visual attention for multi-screen television: measures, toolkit, and experimental findings. Personal and Ubiquitous Computing 19, 5 (2015), 781–801.
[52]
Anqi Wang, Ze Gao, Zeyu Wang, Pan Hui, and Tristan Braud. 2022. Envisioning A Hyper-Learning System in the Age of Metaverse. In Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. 1–2.
[53]
Yafang Wang, Chenglei Yang, Shijun Liu, Rui Wang, and Xiangxu Meng. 2007. Grid-enabled Chinese natural language-based information retrieval system in digital museum. In 2007 2nd International Conference on Pervasive Computing and Applications. IEEE, 594–599.
[54]
Rafal Wojciechowski, Krzysztof Walczak, Martin White, and Wojciech Cellary. 2004. Building virtual and augmented reality museum exhibitions. In Proceedings of the ninth international conference on 3D Web technology. 135–144.
[55]
Terry CW Wong and Wim van der Plas. [n. d.]. Third Summit on New Media Art Archiving. ([n. d.]).
[56]
Wenge Xu, Hai-Ning Liang, Xiaoyue Ma, and Xiang Li. 2020. VirusBoxing: A HIIT-based VR boxing game. In Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play. 98–102.
Index Terms
- VR PreM+: An Immersive Pre-learning Branching Visualization System for Museum Tours
Recommendations
Haptic around: multiple tactile sensations for immersive environment and interaction in virtual reality
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and TechnologyIn this paper, we present Haptic Around, a hybrid-haptic feedback system, which utilizes fan, hot air blower, mist creator and heat light to recreate multiple tactile sensations in virtual reality for enhancing the immersive environment and interaction. ...
A comparison of immersive HMD, fish tank VR and fish tank with haptics displays for volume visualization
APGV '06: Proceedings of the 3rd symposium on Applied perception in graphics and visualizationAlthough a wide range of virtual reality (VR) systems are in use, there are few guidelines to help system and application developers select the components most appropriate for the domain problem they are investigating. Using the results of an empirical ...
FlowZen: Using Hybrid-Haptic and Particle for Enhancing Immersive Experience via Continuous Illusion of Wind
SIGGRAPH '24: ACM SIGGRAPH 2024 Immersive PavilionWith the advancement of haptic technology, we can enable a haptic experience to simulate environmental changes seamlessly, even as the user moves around in the immersive virtual reality world. For instance, stationary non-contact tactile devices, such ...
Comments
Information & Contributors
Information
Published In
November 2023
634 pages
ISBN:9798400716454
DOI:10.1145/3629606
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 February 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- This research was partially supported by the Extended Reality and Immersive Media (XRIM) Lab at HKUST and the National Social Science Foundation of China (20CJY045).
Conference
CHCHI 2023
Acceptance Rates
Overall Acceptance Rate 17 of 40 submissions, 43%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 63Total Downloads
- Downloads (Last 12 months)63
- Downloads (Last 6 weeks)10
Reflects downloads up to 30 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format