Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation

Published: 08 October 2021 Publication History
  • Get Citation Alerts
  • Abstract

    In virtual reality (VR), the virtual scenes are pre-designed by creators. Our physical surroundings, however, comprise significantly varied sizes, layouts, and components. To bridge the gap and further enable natural navigation, recent solutions have been proposed to redirect users or recreate the virtual content. However, they suffer from either interrupted experience or distorted appearances. We present a novel VR-oriented algorithm that automatically restructures a given virtual scene for a user’s physical environment. Different from the previous methods, we introduce neither interrupted walking experience nor curved appearances. Instead, a perception-aware function optimizes our retargeting technique to preserve the fidelity of the virtual scene that appears in VR head-mounted displays. Besides geometric and topological properties, it emphasizes the unique first-person view perceptual factors in VR, such as dynamic visibility and objectwise relationships. We conduct both analytical experiments and subjective studies. The results demonstrate our system’s versatile capability and practicability for natural navigation in VR: It reduces the virtual space by 40% without statistical loss of perceptual identicality.

    Supplementary Material

    dong (dong.zip)
    Supplemental movie, appendix, image and software files for, Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation

    References

    [1]
    Shai Avidan and Ariel Shamir. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3 (2007), 10.
    [2]
    Mahdi Azmandian, Timofey Grechkin, Mark Bolas, and Evan Suma. 2016. The redirected walking toolkit: A unified development platform for exploring large virtual environments. In Proceedings of the IEEE 2nd Workshop on Everyday Virtual Reality (WEVR’16). 9–14.
    [3]
    Mahdi Azmandian, Timofey Grechkin, and Evan Suma Rosenberg. 2017. An evaluation of strategies for two-user redirected walking in shared physical spaces. In Proceedings of the IEEE Conference on Virtual Reality (VR’17). 91–98.
    [4]
    Zhi-Chao Dong, Xiao-Ming Fu, Zeshi Yang, and Ligang Liu. 2019. Redirected smooth mappings for multiuser real walking in virtual reality. ACM Trans. Graph. 38, 5 (2019), 149:1–149:17.
    [5]
    Zhi-Chao Dong, Xiao-Ming Fu, Chi Zhang, Kang Wu, and Ligang Liu. 2017. Smooth assembled mappings for large-scale real walking. ACM Trans. Graph. (SIGGRAPH ASIA) 36, 6 (2017).
    [6]
    Russell A. Epstein and Chris I. Baker. 2019. Scene perception in the human brain. Ann. Rev. Vis. Sci. 5, 1 (2019), 373–397. https://doi.org/10.1146/annurev-vision-091718-014809
    [7]
    Ran Gal, Olga Sorkine, and Daniel Cohen-Or. 2006. Feature-aware texturing. In Eurographics Symposium on Rendering Techniques. 297–303.
    [8]
    Stefan Hartmann, Björn Krüger, and Reinhard Klein. 2015. Content-aware re-targeting of discrete element layouts. In Proceedings of the International Conference on Computer Graphics, Visualization and Computer Vision, Vol. 23. 173–182.
    [9]
    John Henderson, David Zhu, and Christine Larson. 2011. Functions of parahippocampal place area and retrosplenial cortex in real-world scene analysis: An fMRI study. Vis. Cogn. 19, 7 (08 2011), 910–927. https://doi.org/10.1080/13506285.2011.596852
    [10]
    John M. Henderson and Andrew Hollingworth. 1999. High-level scene perception. Ann. Rev. Psychol. 50, 1 (1999), 243–271.
    [11]
    John M. Henderson, Christine L. Larson, and David C. Zhu. 2008. Full scenes produce more activation than close-up scenes and scene-diagnostic objects in parahippocampal and retrosplenial cortex: An fMRI study. Brain Cogn. 66, 1 (2008), 40–49. https://doi.org/10.1016/j.bandc.2007.05.001
    [12]
    Bill Hillier and Alan Penn. 1991. Visible colleges: Structure and randomness in the place of discovery. Sci. Context 4, 1 (1991), 23–50.
    [13]
    Eric Hodgson and Eric Bachmann. 2013. Comparing four approaches to generalized redirected walking: Simulation and live user data. IEEE. T. Vis. Comput. Gr. 19, 4 (2013), 634–643.
    [14]
    Ping Hu, Qi Sun, Piotr Didyk, Li-Yi Wei, and Arie E. Kaufman. 2019. Reducing simulator sickness with perceptual camera control. ACM Trans. Graph. 38, 6, Article 210 (Nov. 2019), 12 pages. https://doi.org/10.1145/3355089.3356490
    [15]
    Chun-Kai Huang, Yi-Ling Chen, I-Chao Shen, and Bing-Yu Chen. 2016. Retargeting 3D objects and scenes with a general framework. Comput. Graph. Forum 35, 7 (2016), 33–42.
    [16]
    Yong Jin, Ligang Liu, and Qingbiao Wu. 2010. Nonhomogeneous scaling optimization for realtime image resizing. Vis. Comput. 26, 6-8 (2010), 769–778.
    [17]
    Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 3 (1993), 203–220.
    [18]
    Talia Konkle, Timothy F. Brady, George A. Alvarez, and Aude Oliva. 2010. Scene memory is more detailed than you think: The role of categories in visual long-term memory. Psychol. Sci. 21, 11 (2010), 1551–1556.
    [19]
    Vladislav Kraevoy, Alla Sheffer, Ariel Shamir, and Daniel Cohen-Or. 2008. Non-homogeneous resizing of complex models. ACM Trans. Graph. 27, 5 (2008), 111.
    [20]
    Eike Langbehn and Frank Steinicke. 2018. Redirected Walking in Virtual Reality. Springer Encyclopedia of Computer Graphics and Games, 1–11.
    [21]
    Eike Langbehn, Frank Steinicke, Markus Lappe, Gregory F. Welch, and Gerd Bruder. 2018. In the blink of an eye: Leveraging blink-induced suppression for imperceptible position and orientation redirection in virtual reality. ACM Trans. Graph. 37, 4 (2018), 66:1–66:11.
    [22]
    Jinjie Lin, Daniel Cohen-Or, Hao Zhang, Cheng Liang, Andrei Sharf, Oliver Deussen, and Baoquan Chen. 2011. Structure-preserving retargeting of irregular 3D architecture. ACM Trans. Graph. 30, 6 (2011), 183:1–183:10.
    [23]
    Alex Mansfield, Peter Gehler, Luc Van Gool, and Carsten Rother. 2010. Scene carving: Scene consistent image retargeting. In Proceedings of the European Conference on Computer Vision. 143–156.
    [24]
    Sebastian Marwecki and Patrick Baudisch. 2018. Scenograph: Fitting real-walking vr experiences into various tracking volumes. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST’18). Association for Computing Machinery, New York, NY, 511–520. https://doi.org/10.1145/3242587.3242648
    [25]
    Edvard I. Moser, Emilio Kropff, and May-Britt Moser. 2008. Place cells, grid cells, and the brain’s spatial representation system. Annu. Rev. Neurosci. 31, 1 (2008), 69–89.
    [26]
    Niels Christian Nilsson, Tabitha Peck, Gerd Bruder, Eri Hodgson, Stefania Serafin, Mary Whitton, Frank Steinicke, and Evan Suma Rosenberg. 2018. 15 years of research on redirected walking in immersive virtual environments. IEEE Comput. Graph. Appl. 38, 2 (2018), 44–56.
    [27]
    Aude Oliva and Antonio Torralba. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 3 (2001), 145–175.
    [28]
    Nitish Padmanaban, Timon Ruban, Vincent Sitzmann, Anthony M. Norcia, and Gordon Wetzstein. 2018. Towards a machine-learning approach for sickness prediction in 360circ stereoscopic videos. IEEE Trans. Visual. Comput. Graph. 24, 4 (2018), 1594–1603.
    [29]
    Soojin Park, Talia Konkle, and Aude Oliva. 2014. Parametric coding of the size and clutter of natural scenes in the human brain. Cerebr. Cortex 25, 7 (01 2014), 1792–1805. https://doi.org/10.1093/cercor/bht418arXiv:https://academic. oup.com/cercor/article-pdf/25/7/1792/14102860/bht 418.pdf.
    [30]
    John Peponis, Ruth Conroy Dalton, Jean Wineman, and Nick Dalton. 2004. Measuring the effects of layout upon visitors’ spatial behaviors in open plan exhibition settings. Environ. Plan. B: Plan. Des. 31, 3 (2004), 453–473.
    [31]
    Michael Rubinstein, Diego Gutierrez, Olga Sorkine, and Ariel Shamir. 2010. A comparative study of image retargeting. ACM Trans. Graph. 29, 6 (2010), 160.
    [32]
    Michael Rubinstein, Ariel Shamir, and Shai Avidan. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3 (2008), 16.
    [33]
    Michael Rubinstein, Ariel Shamir, and Shai Avidan. 2009. Multi-operator media retargeting. ACM Trans. Graph. 28, 3 (2009), 23.
    [34]
    Ariel Shamir and Olga Sorkine. 2009. Visual media retargeting. In ACM SIGGRAPH ASIA 2009 Courses. 11.
    [35]
    Evan A. Suma, Seth Clark, David Krum, Samantha Finkelstein, Mark Bolas, and Zachary Warte. 2011. Leveraging change blindness for redirection in virtual environments. In Proceedings of the IEEE Conference on Virtual Reality (VR’11). 159–166.
    [36]
    Evan A. Suma, Zachary Lipps, Samantha Finkelstein, David M. Krum, and Mark Bolas. 2012. Impossible spaces: Maximizing natural walking in virtual environments with self-overlapping architecture. IEEE. T. Vis. Comput. Gr. 18, 4 (2012), 555–564.
    [37]
    Qi Sun, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan Mcguire, David Luebke, and Arie Kaufman. 2018. Towards virtual reality infinite walking: dynamic saccadic redirection. ACM Trans. Graph. 37, 4 (2018), 67:1–67:13.
    [38]
    Qi Sun, Li-Yi Wei, and Arie Kaufman. 2016. Mapping virtual and physical reality. ACM Trans. Graph. (SIGGRAPH) 35, 4 (2016), 64:1–64:12.
    [39]
    Daniel Vaquero, Matthew Turk, Kari Pulli, Marius Tico, and Natasha Gelfand. 2010. A survey of image retargeting techniques. In Applications of Digital Image Processing XXXIII, Vol. 7798. 328–342.
    [40]
    K. Vasylevska and H. Kaufmann. 2017. Towards efficient spatial compression in self-overlapping virtual environments. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI’17). 12–21. https://doi.org/10.1109/3DUI.2017.7893312
    [41]
    Khrystyna Vasylevska, Hannes Kaufmann, Mark Bolas, and Evan A. Suma. 2013. Flexible spaces: Dynamic layout generation for infinite walking in virtual environments. In Proceedings of the 2013 IEEE Symposium on 3D User Interfaces (3DUI’13). 39–42.
    [42]
    Shu-Fan Wang and Shang-Hong Lai. 2009. Fast structure-preserving image retargeting. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 1049–1052.
    [43]
    Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5 (2008), 118.
    [44]
    Lap-Fai Yu, Sai-Kit Yeung, Chi-Keung Tang, Demetri Terzopoulos, Tony F. Chan, and Stanley J. Osher. 2011. Make it home: Automatic optimization of furniture arrangement. ACM Trans. Graph. 30, 4, Article 86 (Jul. 2011), 12 pages. https://doi.org/10.1145/2010324.1964981
    [45]
    Gregory Zelinsky. 2013. Understanding scene understanding. Front. Psychol. 4 (2013), 954. https://doi.org/10.3389/fpsyg.2013.00954
    [46]
    Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2014. Learning deep features for scene recognition using places database. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). 487–495.

    Cited By

    View all
    • (2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
    • (2023)Edge-Centric Space Rescaling with Redirected Walking for Dissimilar Physical-Virtual Space Registration2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00098(829-838)Online publication date: 16-Oct-2023
    • (2023)SMigraPH: a perceptually retained method for passive haptics-based migration of MR indoor scenesThe Visual Computer10.1007/s00371-023-03220-2Online publication date: 30-Dec-2023
    • Show More Cited By

    Index Terms

    1. Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR Navigation

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 40, Issue 5
      October 2021
      190 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3477320
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 08 October 2021
      Accepted: 01 June 2021
      Revised: 01 April 2021
      Received: 2021
      Published in TOG Volume 40, Issue 5

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Virtual reality
      2. perception
      3. retargeting

      Qualifiers

      • Research-article
      • Refereed

      Funding Sources

      • National Natural Science Foundation of China
      • Zhejiang Lab
      • USTC Research Funds of Double First-Class Initiative

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)153
      • Downloads (Last 6 weeks)11
      Reflects downloads up to 11 Aug 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
      • (2023)Edge-Centric Space Rescaling with Redirected Walking for Dissimilar Physical-Virtual Space Registration2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00098(829-838)Online publication date: 16-Oct-2023
      • (2023)SMigraPH: a perceptually retained method for passive haptics-based migration of MR indoor scenesThe Visual Computer10.1007/s00371-023-03220-2Online publication date: 30-Dec-2023
      • (2023)Real-scene-constrained virtual scene layout synthesis for mixed realityThe Visual Computer10.1007/s00371-023-03167-440:9(6319-6339)Online publication date: 12-Dec-2023
      • (2022)Dynamic optimal space partitioning for redirected walking in multi-user environmentACM Transactions on Graphics10.1145/3528223.353011341:4(1-14)Online publication date: 22-Jul-2022
      • (2022)Interactive augmented reality storytelling guided by scene semanticsACM Transactions on Graphics10.1145/3528223.353006141:4(1-15)Online publication date: 22-Jul-2022
      • (2022)ScalAR: Authoring Semantically Adaptive Augmented Reality Experiences in Virtual RealityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517665(1-18)Online publication date: 29-Apr-2022
      • (2022)Foldable Spaces: An Overt Redirection Approach for Natural Walking in Virtual Reality2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR51125.2022.00035(167-175)Online publication date: Mar-2022
      • (2022) 18‐1: Invited Paper: Modeling and Optimizing Human‐in‐the‐Loop Visual Perception Using Immersive Displays: A Review SID Symposium Digest of Technical Papers10.1002/sdtp.1545053:1(190-193)Online publication date: 28-Jun-2022

      View Options

      Get Access

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media