research-article

Open access

Reverse Pass-Through VR

Authors:

Nathan Matsuda,

Brian Wheelwright,

Douglas LanmanAuthors Info & Claims

SIGGRAPH '21: ACM SIGGRAPH 2021 Emerging Technologies

Article No.: 13, Pages 1 - 4

https://doi.org/10.1145/3450550.3465338

Published: 06 August 2021 Publication History

All formats PDF

Abstract

We introduce reverse pass-through VR, wherein a three-dimensional view of the wearer’s eyes is presented to multiple outside viewers in a perspective-correct manner, with a prototype headset containing a world-facing light field display. This approach, in conjunction with existing video (forward) pass-through technology, enables more seamless interactions between people with and without headsets in social or professional contexts. Reverse pass-through VR ties together research in social telepresence and copresence, autostereoscopic displays, and facial capture to enable natural eye contact and other important non-verbal cues in a wider range of interaction scenarios.

Supplementary Material

VTT File (3450550.3465338.vtt)

Download
15.53 KB

MP4 File (3450550.3465338.mp4)

Presentation video.

Download
958.87 MB

References

[1]

Oleg Alexander, Mike Rogers, William Lambeth, Matt Chiang, and Paul Debevec. 2009. The Digital Emily Project: Photoreal Facial Modeling and Animation. In ACM SIGGRAPH Courses. 1–15.

[2]

Blender Foundation. 2021. Blender - a 3D modelling and rendering package. Blender Foundation. http://www.blender.org

[3]

Liwei Chan and Kouta Minamizawa. 2017. FrontFace: Facilitating Communication between HMD Users and Outsiders Using Front-Facing-Screen HMDs. In ACM Human-Computer Interaction with Mobile Devices and Services. Article 22.

[4]

Ying Geng, Jacques Gollier, Brian Wheelwright, Fenglin Peng, Yusufu Sulai, Brant Lewis, Ning Chan, Wai Sze Tiffany Lam, Alexander Fix, Douglas Lanman, 2018. Viewing Optics for Immersive Near-Eye Displays: Pupil Swim / Size and Weight / Stray Light. In Digital Optics for Immersive Displays, Vol. 10676. International Society for Optics and Photonics.

[5]

Jan Gugenheimer, Christian Mai, Mark McGill, Julie Williamson, Frank Steinicke, and Ken Perlin. 2019. Challenges Using Head-Mounted Displays in Shared and Social Spaces. In ACM Conference on Human Factors in Computing Systems Extended Abstracts.

[6]

Kopin. 2021. Kopin’s 2.6k x 2.6k OLED Display Incorporated in Panasonic’s New VR Glasses. (Jan 2021). https://kopin.irpass.com/profiles/investor/NewsPrint.asp?v=6&b=2379&ID=96490&m=rl&g=1207

[7]

Stephen Lombardi, Jason Saragih, Tomas Simon, and Yaser Sheikh. 2018. Deep Appearance Models for Face Rendering. ACM Transactions on Graphics 37, 4, Article 68 (July 2018).

Digital Library

[8]

Christian Mai and Mohamed Khamis. 2018. Public HMDs: Modeling and Understanding User Behavior around Public Head-Mounted Displays. In ACM Pervasive Displays. Article 21.

[9]

Christian Mai, Lukas Rambold, and Mohamed Khamis. 2017. TransparentHMD: Revealing the HMD User’s Face to Bystanders. In ACM Mobile and Ubiquitous Multimedia.

[10]

Andrew Maimone and Junren Wang. 2020. Holographic Optics for Thin and Lightweight Virtual Reality. ACM Transactions on Graphics 39, 4 (2020).

Digital Library

[11]

Manuel Martínez-Corral and Bahram Javidi. 2018. Fundamentals of 3D Imaging and Displays: A Tutorial on Integral Imaging, Light-Field, and Plenoptic Systems. OSA Advances in Optics and Photonics 10, 3 (Sep 2018), 512–566.

[12]

Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. 2016. A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. In IEEE Conference on Computer Vision and Pattern Recognition.

[13]

Valentin Schwind, Jens Reinhardt, Rufat Rzayev, Niels Henze, and Katrin Wolf. 2018. Virtual Reality on the Go? A Study on Social Acceptance of VR Glasses. In ACM Human-Computer Interaction with Mobile Devices and Services. 111–118.

[14]

Yan Wang, Zihang Lai, Gao Huang, Brian H Wang, Laurens Van Der Maaten, Mark Campbell, and Kilian Q Weinberger. 2019. Anytime Stereo Image Depth Estimation on Mobile Devices. In IEEE International Conference on Robotics and Automation.

[15]

Timothy L Wong, Zhisheng Yun, Gregg Ambur, and Jo Etter. 2017. Folded Optics with Birefringent Reflective Polarizers. In Digital Optical Technologies, Vol. 10335. International Society for Optics and Photonics.

[16]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In IEEE International Conference on Computer Vision.

Cited By

Bozgeyikli EBozgeyikli LGomes V(2024)Googly Eyes: Exploring Effects of Displaying User’s Eyes Outward on a Virtual Reality Head-Mounted Display on User Experience2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00117(979-989)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00117
Combe TFribourg RDetto LNorm J(2024)Exploring the Influence of Virtual Avatar Heads in Mixed Reality on Social Presence, Performance and User Experience in Collaborative TasksIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337205130:5(2206-2216)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372051
Kang HPark MJang JKim DKoo GJung GKim HKim GYang JYoon S(2024)70‐1: A New Approach to High‐Resolution Light Field Display for Higher Realistic ExpressionSID Symposium Digest of Technical Papers10.1002/sdtp.1769555:1(961-964)Online publication date: 30-Jul-2024
https://doi.org/10.1002/sdtp.17695
Show More Cited By

Recommendations

VR social copresence with light field displays

As virtual reality (VR) devices become increasingly commonplace, asymmetric interactions between people with and without headsets are becoming more frequent. Existing video pass-through VR headsets solve one side of these asymmetric interactions by ...
Near-eye light field displays

We propose near-eye light field displays that enable thin, lightweight head-mounted displays (HMDs) capable of presenting nearly correct convergence, accommodation, binocular disparity, and retinal defocus depth cues. Sharp images are depicted by out-of-...
Near-eye light field displays
SIGGRAPH '13: ACM SIGGRAPH 2013 Talks

We propose a light-field-based approach to near-eye display that allows for thin, lightweight head-mounted displays capable of depicting accurate accommodation, convergence, and binocular disparity depth cues. Our near-eye light field displays depict ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '21: ACM SIGGRAPH 2021 Emerging Technologies

August 2021

53 pages

ISBN:9781450383646

DOI:10.1145/3450550

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGGRAPH '21

Sponsor:

SIGGRAPH

SIGGRAPH '21: Special Interest Group on Computer Graphics and Interactive Techniques Conference

August 9 - 13, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
2,046
Total Downloads

Downloads (Last 12 months)409
Downloads (Last 6 weeks)49

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bozgeyikli EBozgeyikli LGomes V(2024)Googly Eyes: Exploring Effects of Displaying User’s Eyes Outward on a Virtual Reality Head-Mounted Display on User Experience2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00117(979-989)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00117
Combe TFribourg RDetto LNorm J(2024)Exploring the Influence of Virtual Avatar Heads in Mixed Reality on Social Presence, Performance and User Experience in Collaborative TasksIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337205130:5(2206-2216)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372051
Kang HPark MJang JKim DKoo GJung GKim HKim GYang JYoon S(2024)70‐1: A New Approach to High‐Resolution Light Field Display for Higher Realistic ExpressionSID Symposium Digest of Technical Papers10.1002/sdtp.1769555:1(961-964)Online publication date: 30-Jul-2024
https://doi.org/10.1002/sdtp.17695
Auda JGruenefeld UFaltaous SMayer SSchneegass S(2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3616536
Bozgeyikli EGomes V(2022)Googly Eyes: Displaying User's Eyes on a Head-Mounted Display for Improved Nonverbal CommunicationExtended Abstracts of the 2022 Annual Symposium on Computer-Human Interaction in Play10.1145/3505270.3558348(253-260)Online publication date: 2-Nov-2022
https://dl.acm.org/doi/10.1145/3505270.3558348
Gottsacker MSyamil RWisniewski PBruder GCruz-Neira CWelch G(2022)Exploring Cues and Signaling to Improve Cross-Reality Interruptions2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00179(827-832)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00179
Suomalainen MSakcak BWidagdo AKalliokoski JMimnaugh KChambers AOjala TLaValle S(2022)Unwinding Rotations Improves User Comfort with Immersive Telepresence Robots2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889388(511-520)Online publication date: 7-Mar-2022
https://doi.org/10.1109/HRI53351.2022.9889388
Matsuda NWheelwright BHegland JLanman D(2021)VR social copresence with light field displaysACM Transactions on Graphics10.1145/3478513.348048140:6(1-13)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480481

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten