Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation

Published: 08 October 2021 Publication History

Abstract

In virtual reality (VR), the virtual scenes are pre-designed by creators. Our physical surroundings, however, comprise significantly varied sizes, layouts, and components. To bridge the gap and further enable natural navigation, recent solutions have been proposed to redirect users or recreate the virtual content. However, they suffer from either interrupted experience or distorted appearances. We present a novel VR-oriented algorithm that automatically restructures a given virtual scene for a user’s physical environment. Different from the previous methods, we introduce neither interrupted walking experience nor curved appearances. Instead, a perception-aware function optimizes our retargeting technique to preserve the fidelity of the virtual scene that appears in VR head-mounted displays. Besides geometric and topological properties, it emphasizes the unique first-person view perceptual factors in VR, such as dynamic visibility and objectwise relationships. We conduct both analytical experiments and subjective studies. The results demonstrate our system’s versatile capability and practicability for natural navigation in VR: It reduces the virtual space by 40% without statistical loss of perceptual identicality.

Supplementary Material

dong (dong.zip)
Supplemental movie, appendix, image and software files for, Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation

References

[1]
Shai Avidan and Ariel Shamir. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3 (2007), 10.
[2]
Mahdi Azmandian, Timofey Grechkin, Mark Bolas, and Evan Suma. 2016. The redirected walking toolkit: A unified development platform for exploring large virtual environments. In Proceedings of the IEEE 2nd Workshop on Everyday Virtual Reality (WEVR’16). 9–14.
[3]
Mahdi Azmandian, Timofey Grechkin, and Evan Suma Rosenberg. 2017. An evaluation of strategies for two-user redirected walking in shared physical spaces. In Proceedings of the IEEE Conference on Virtual Reality (VR’17). 91–98.
[4]
Zhi-Chao Dong, Xiao-Ming Fu, Zeshi Yang, and Ligang Liu. 2019. Redirected smooth mappings for multiuser real walking in virtual reality. ACM Trans. Graph. 38, 5 (2019), 149:1–149:17.
[5]
Zhi-Chao Dong, Xiao-Ming Fu, Chi Zhang, Kang Wu, and Ligang Liu. 2017. Smooth assembled mappings for large-scale real walking. ACM Trans. Graph. (SIGGRAPH ASIA) 36, 6 (2017).
[6]
Russell A. Epstein and Chris I. Baker. 2019. Scene perception in the human brain. Ann. Rev. Vis. Sci. 5, 1 (2019), 373–397. https://doi.org/10.1146/annurev-vision-091718-014809
[7]
Ran Gal, Olga Sorkine, and Daniel Cohen-Or. 2006. Feature-aware texturing. In Eurographics Symposium on Rendering Techniques. 297–303.
[8]
Stefan Hartmann, Björn Krüger, and Reinhard Klein. 2015. Content-aware re-targeting of discrete element layouts. In Proceedings of the International Conference on Computer Graphics, Visualization and Computer Vision, Vol. 23. 173–182.
[9]
John Henderson, David Zhu, and Christine Larson. 2011. Functions of parahippocampal place area and retrosplenial cortex in real-world scene analysis: An fMRI study. Vis. Cogn. 19, 7 (08 2011), 910–927. https://doi.org/10.1080/13506285.2011.596852
[10]
John M. Henderson and Andrew Hollingworth. 1999. High-level scene perception. Ann. Rev. Psychol. 50, 1 (1999), 243–271.
[11]
John M. Henderson, Christine L. Larson, and David C. Zhu. 2008. Full scenes produce more activation than close-up scenes and scene-diagnostic objects in parahippocampal and retrosplenial cortex: An fMRI study. Brain Cogn. 66, 1 (2008), 40–49. https://doi.org/10.1016/j.bandc.2007.05.001
[12]
Bill Hillier and Alan Penn. 1991. Visible colleges: Structure and randomness in the place of discovery. Sci. Context 4, 1 (1991), 23–50.
[13]
Eric Hodgson and Eric Bachmann. 2013. Comparing four approaches to generalized redirected walking: Simulation and live user data. IEEE. T. Vis. Comput. Gr. 19, 4 (2013), 634–643.
[14]
Ping Hu, Qi Sun, Piotr Didyk, Li-Yi Wei, and Arie E. Kaufman. 2019. Reducing simulator sickness with perceptual camera control. ACM Trans. Graph. 38, 6, Article 210 (Nov. 2019), 12 pages. https://doi.org/10.1145/3355089.3356490
[15]
Chun-Kai Huang, Yi-Ling Chen, I-Chao Shen, and Bing-Yu Chen. 2016. Retargeting 3D objects and scenes with a general framework. Comput. Graph. Forum 35, 7 (2016), 33–42.
[16]
Yong Jin, Ligang Liu, and Qingbiao Wu. 2010. Nonhomogeneous scaling optimization for realtime image resizing. Vis. Comput. 26, 6-8 (2010), 769–778.
[17]
Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 3 (1993), 203–220.
[18]
Talia Konkle, Timothy F. Brady, George A. Alvarez, and Aude Oliva. 2010. Scene memory is more detailed than you think: The role of categories in visual long-term memory. Psychol. Sci. 21, 11 (2010), 1551–1556.
[19]
Vladislav Kraevoy, Alla Sheffer, Ariel Shamir, and Daniel Cohen-Or. 2008. Non-homogeneous resizing of complex models. ACM Trans. Graph. 27, 5 (2008), 111.
[20]
Eike Langbehn and Frank Steinicke. 2018. Redirected Walking in Virtual Reality. Springer Encyclopedia of Computer Graphics and Games, 1–11.
[21]
Eike Langbehn, Frank Steinicke, Markus Lappe, Gregory F. Welch, and Gerd Bruder. 2018. In the blink of an eye: Leveraging blink-induced suppression for imperceptible position and orientation redirection in virtual reality. ACM Trans. Graph. 37, 4 (2018), 66:1–66:11.
[22]
Jinjie Lin, Daniel Cohen-Or, Hao Zhang, Cheng Liang, Andrei Sharf, Oliver Deussen, and Baoquan Chen. 2011. Structure-preserving retargeting of irregular 3D architecture. ACM Trans. Graph. 30, 6 (2011), 183:1–183:10.
[23]
Alex Mansfield, Peter Gehler, Luc Van Gool, and Carsten Rother. 2010. Scene carving: Scene consistent image retargeting. In Proceedings of the European Conference on Computer Vision. 143–156.
[24]
Sebastian Marwecki and Patrick Baudisch. 2018. Scenograph: Fitting real-walking vr experiences into various tracking volumes. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST’18). Association for Computing Machinery, New York, NY, 511–520. https://doi.org/10.1145/3242587.3242648
[25]
Edvard I. Moser, Emilio Kropff, and May-Britt Moser. 2008. Place cells, grid cells, and the brain’s spatial representation system. Annu. Rev. Neurosci. 31, 1 (2008), 69–89.
[26]
Niels Christian Nilsson, Tabitha Peck, Gerd Bruder, Eri Hodgson, Stefania Serafin, Mary Whitton, Frank Steinicke, and Evan Suma Rosenberg. 2018. 15 years of research on redirected walking in immersive virtual environments. IEEE Comput. Graph. Appl. 38, 2 (2018), 44–56.
[27]
Aude Oliva and Antonio Torralba. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 3 (2001), 145–175.
[28]
Nitish Padmanaban, Timon Ruban, Vincent Sitzmann, Anthony M. Norcia, and Gordon Wetzstein. 2018. Towards a machine-learning approach for sickness prediction in 360circ stereoscopic videos. IEEE Trans. Visual. Comput. Graph. 24, 4 (2018), 1594–1603.
[29]
Soojin Park, Talia Konkle, and Aude Oliva. 2014. Parametric coding of the size and clutter of natural scenes in the human brain. Cerebr. Cortex 25, 7 (01 2014), 1792–1805. https://doi.org/10.1093/cercor/bht418arXiv:https://academic. oup.com/cercor/article-pdf/25/7/1792/14102860/bht 418.pdf.
[30]
John Peponis, Ruth Conroy Dalton, Jean Wineman, and Nick Dalton. 2004. Measuring the effects of layout upon visitors’ spatial behaviors in open plan exhibition settings. Environ. Plan. B: Plan. Des. 31, 3 (2004), 453–473.
[31]
Michael Rubinstein, Diego Gutierrez, Olga Sorkine, and Ariel Shamir. 2010. A comparative study of image retargeting. ACM Trans. Graph. 29, 6 (2010), 160.
[32]
Michael Rubinstein, Ariel Shamir, and Shai Avidan. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3 (2008), 16.
[33]
Michael Rubinstein, Ariel Shamir, and Shai Avidan. 2009. Multi-operator media retargeting. ACM Trans. Graph. 28, 3 (2009), 23.
[34]
Ariel Shamir and Olga Sorkine. 2009. Visual media retargeting. In ACM SIGGRAPH ASIA 2009 Courses. 11.
[35]
Evan A. Suma, Seth Clark, David Krum, Samantha Finkelstein, Mark Bolas, and Zachary Warte. 2011. Leveraging change blindness for redirection in virtual environments. In Proceedings of the IEEE Conference on Virtual Reality (VR’11). 159–166.
[36]
Evan A. Suma, Zachary Lipps, Samantha Finkelstein, David M. Krum, and Mark Bolas. 2012. Impossible spaces: Maximizing natural walking in virtual environments with self-overlapping architecture. IEEE. T. Vis. Comput. Gr. 18, 4 (2012), 555–564.
[37]
Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan Mcguire, David Luebke, and Arie Kaufman. 2018. Towards virtual reality infinite walking: dynamic saccadic redirection. ACM Trans. Graph. 37, 4 (2018), 67:1–67:13.
[38]
Qi Sun, Li-Yi Wei, and Arie Kaufman. 2016. Mapping virtual and physical reality. ACM Trans. Graph. (SIGGRAPH) 35, 4 (2016), 64:1–64:12.
[39]
Daniel Vaquero, Matthew Turk, Kari Pulli, Marius Tico, and Natasha Gelfand. 2010. A survey of image retargeting techniques. In Applications of Digital Image Processing XXXIII, Vol. 7798. 328–342.
[40]
K. Vasylevska and H. Kaufmann. 2017. Towards efficient spatial compression in self-overlapping virtual environments. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI’17). 12–21. https://doi.org/10.1109/3DUI.2017.7893312
[41]
Khrystyna Vasylevska, Hannes Kaufmann, Mark Bolas, and Evan A. Suma. 2013. Flexible spaces: Dynamic layout generation for infinite walking in virtual environments. In Proceedings of the 2013 IEEE Symposium on 3D User Interfaces (3DUI’13). 39–42.
[42]
Shu-Fan Wang and Shang-Hong Lai. 2009. Fast structure-preserving image retargeting. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 1049–1052.
[43]
Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5 (2008), 118.
[44]
Lap-Fai Yu, Sai-Kit Yeung, Chi-Keung Tang, Demetri Terzopoulos, Tony F. Chan, and Stanley J. Osher. 2011. Make it home: Automatic optimization of furniture arrangement. ACM Trans. Graph. 30, 4, Article 86 (Jul. 2011), 12 pages. https://doi.org/10.1145/2010324.1964981
[45]
Gregory Zelinsky. 2013. Understanding scene understanding. Front. Psychol. 4 (2013), 954. https://doi.org/10.3389/fpsyg.2013.00954
[46]
Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2014. Learning deep features for scene recognition using places database. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). 487–495.

Cited By

View all
  • (2024)Investigation of Redirection Algorithms in Small Tracking SpacesProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687749(1-9)Online publication date: 9-Oct-2024
  • (2024)MineXR: Mining Personalized Extended Reality InterfacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642394(1-17)Online publication date: 11-May-2024
  • (2024)Sicknificant Steps: A Systematic Review and Meta-analysis of VR Sickness in Walking-based Locomotion for Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641974(1-36)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 40, Issue 5
    October 2021
    190 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/3477320
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 October 2021
    Accepted: 01 June 2021
    Revised: 01 April 2021
    Received: 2021
    Published in TOG Volume 40, Issue 5

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Virtual reality
    2. perception
    3. retargeting

    Qualifiers

    • Research-article
    • Refereed

    Funding Sources

    • National Natural Science Foundation of China
    • Zhejiang Lab
    • USTC Research Funds of Double First-Class Initiative

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)151
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 21 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Investigation of Redirection Algorithms in Small Tracking SpacesProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687749(1-9)Online publication date: 9-Oct-2024
    • (2024)MineXR: Mining Personalized Extended Reality InterfacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642394(1-17)Online publication date: 11-May-2024
    • (2024)Sicknificant Steps: A Systematic Review and Meta-analysis of VR Sickness in Walking-based Locomotion for Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641974(1-36)Online publication date: 11-May-2024
    • (2024)Overcoming Spatial Constraints in VR: A Survey of Redirected Walking TechniquesJournal of Computer Science and Technology10.1007/s11390-024-4585-339:4(841-870)Online publication date: 20-Sep-2024
    • (2024)Real-scene-constrained virtual scene layout synthesis for mixed realityThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03167-440:9(6319-6339)Online publication date: 1-Sep-2024
    • (2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
    • (2023)Edge-Centric Space Rescaling with Redirected Walking for Dissimilar Physical-Virtual Space Registration2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00098(829-838)Online publication date: 16-Oct-2023
    • (2023)SMigraPH: a perceptually retained method for passive haptics-based migration of MR indoor scenesThe Visual Computer10.1007/s00371-023-03220-2Online publication date: 30-Dec-2023
    • (2022)Dynamic optimal space partitioning for redirected walking in multi-user environmentACM Transactions on Graphics10.1145/3528223.353011341:4(1-14)Online publication date: 22-Jul-2022
    • (2022)Interactive augmented reality storytelling guided by scene semanticsACM Transactions on Graphics10.1145/3528223.353006141:4(1-15)Online publication date: 22-Jul-2022
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media