research-article

Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment

Authors:

Tomislav Pejsa,

Andrew WilsonAuthors Info & Claims

CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

Pages 1716 - 1725

https://doi.org/10.1145/2818048.2819965

Published: 27 February 2016 Publication History

Abstract

Room2Room is a telepresence system that leverages projected augmented reality to enable life-size, co-present interaction between two remote participants. Our solution recreates the experience of a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their life-size virtual copy into the remote space. This creates an illusion of the remote person's physical presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze, pointing.) In addition to the technical details of two prototype implementations, we contribute strategies for projecting remote participants onto physically plausible locations, such that they form a natural and consistent conversational formation with the local participant. We also present observations and feedback from an evaluation with 7 pairs of participants on the usability of our solution for solving a collaborative, physical task.

Supplementary Material

pejsa.mp4 (1716_pejsa_supmat.mp4)

Pejsa Supplemental Movie

Download
49.22 MB

References

[1]

Steve Benford, Chris Greenhalgh, Gail Reynard, Chris Brown, and Boriana Koleva. 1998. Understanding and constructing shared spaces with mixed-reality boundaries. ACM Trans. Comput.-Hum. Interact. 5, 3 (Sept. 1998), 185-223.

Digital Library

[2]

Hrvoje Benko, Ricardo Jota, and Andrew Wilson. 2012. MirageTable: freehand interaction on a projected augmented reality tabletop. In Proc. ACM CHI '12.199-208.

Digital Library

[3]

Hrvoje Benko, Andrew Wilson, and Federico Zannier. 2014. Dyadic projected spatial augmented reality. In Proc. ACM UIST '14. 645-655.

Digital Library

[4]

Judee K. Burgoon, David B. Buller, Jerold L. Hale, and Mark A. de Turck. 1984. Relational messages associated with nonverbal behaviors. Human Communication Research 10, 3 (1984), 351-378.

[5]

Bill Buxton. 2009. Mediaspace – Meaningspace – Meetingspace. Harrison, S. (Ed) Media Space 20 + Years of Mediated Life, Springer.

[6]

James Carifio and Rocco Perla. 2008. Resolving the 50year debate around using and misusing Likert scales. Medical Education 42, 12 (2008), 1150-1152.

[7]

Chris Greenhalgh and Steven Benford. 1995. MASSIVE: a collaborative virtual environment for teleconferencing. ACM Trans. Comput.-Hum. Interact. 2, 3 (Sept. 1995), 239-261.

Digital Library

[8]

Markus Gross, Stephan Würmlin, Martin Naef, Edouard Lamboray, Christian Spagno, Andreas Kunz, Esther Koller-Meier, Tomas Svoboda, Luc Van Gool, Silke Lang, Kai Strehlke, Andrew Vande Moere, and Oliver Staadt. 2003. blue-c: a spatially immersive display and 3D video portal for telepresence. In Proc. ACM SIGGRAPH '03. 819-827.

Digital Library

[9]

Yusuke Ichikawa, Ken-ichi Okada, Giseok Jeong, Shunsuke Tanaka, and Yutaka Matsushita. 1995. MAJIC videoconferencing system: experiments, evaluation and improvement. In Proc. ECSCW '95. 279-292.

Digital Library

[10]

Hiroshi Ishii and Minoru Kobayashi. 1992. ClearBoard: a seamless medium for shared drawing and conversation with eye contact. In Proc. ACM CHI '92. 525-532.

Digital Library

[11]

Andrew Jones, Magnus Lang, Graham Fyffe, Xueming Yu, Jay Busch, Ian McDowall, Mark Bolas, and Paul Debevec. 2009. Achieving eye contact in a one-to-many 3D video teleconferencing system. ACM Trans. Graph. 28, 3 (2009).

Digital Library

[12]

Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntryre, Nikunj Raghuvanshi, and Lior Shapira. 2014. RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In Proc. ACM UIST '14.

Digital Library

[13]

Sasa Junuzovic, Kori Inkpen, John Tang, Mara Sedlins, and Kristie Fisher. 2012. To see or not to see: a study comparing four-way avatar, video, and audio conferencing for work. In Proc. ACM GROUP '12. 31-34.

Digital Library

[14]

Nicolas Lehment, Daniel Merget, and Gerhard Rigoll. 2014. Creating automatically aligned consensus realities for AR videoconferencing. In Proc. IEEE ISMAR '14.

[15]

Andrew Maimone and Henry Fuchs. 2011. Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras. In Proc. IEEE ISMAR '11. 137-146.

Digital Library

[16]

Andrew Maimone, Xubo Yang, Nate Dierk, Andrei State, Mingsong Dou, and Henry Fuchs. 2013. Generalpurpose telepresence with head-worn optical seethrough displays and projector-based lighting. In Proc. IEEE VR '13. 23-26.

[17]

David T. Nguyen and John Canny. 2009. More than face-to-face: empathy effects of video framing. In Proc. ACM CHI '09. 423-432.

Digital Library

[18]

Ramesh Raskar, Greg Welch, Matt Cutts, Adam Lake, Lev Stesin, and Henry Fuchs. 1998. The office of the future: a unified approach to image-based modeling and spatially immersive displays. In Proc. ACM SIGGRAPH '98. 179-188.

Digital Library

[19]

Abigail J. Sellen. 1992. Speech patterns in videomediated conversations. In Proc. ACM CHI '92. 49-59.

Digital Library

[20]

John Short, Ederyn Williams, and Bruce Christie. 1976. The Social Psychology of Telecommunications. Wiley, New York, NY.

[21]

Mel Slater. 1999. Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire. Presence: Teleoper. Virtual Environ. 8, 5 (Oct. 1999), 560565.

Digital Library

[22]

Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proc. ACM CHI '13. 179-188.

Digital Library

[23]

John C. Tang and Scott Minneman. 1991. VideoWhiteboard: video shadows to support remote collaboration. In Proc. ACM CHI '91. 315-322.

Digital Library

[24]

Martin Usoh, Ernest Catena, Sima Arman, and Mel Slater. 2000. Using Presence Questionnaires in Reality. Presence: Teleoper. Virtual Environ. 9(5) (Oct. 2000), 497-503.

Digital Library

[25]

Pierre Wellner. 1993. Interacting with paper on the DigitalDesk. Commun. ACM 36, 7 (1993), 87-96.

Digital Library

[26]

Andrew D. Wilson, Hrvoje Benko, Shahram Izadi, and Otmar Hilliges. 2012. Steerable augmented reality with the Beamatron. In Proc. ACM UIST '12. 413-422.

Digital Library

[27]

Jakob Zillner, Christoph Rhemann, Shahram Izadi, and Michael Haller. 2014. 3D-Board: A shared workspace featuring remote 3D virtual embodiments. In Proc. ACM UIST '14. 471-479.

Digital Library

Cited By

IWAI D(2024)Projection mapping technologies: A review of current trends and future directionsProceedings of the Japan Academy, Series B10.2183/pjab.100.012100:3(234-251)Online publication date: 11-Mar-2024
https://doi.org/10.2183/pjab.100.012
Rajaram S(2024)Enabling Safer Augmented Reality Experiences: Usable Privacy Interventions for AR Creators and End-UsersAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686708(1-8)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686708
Schuessler MHormann LDachselt RBlake ARother C(2024)Gazing Heads: Investigating Gaze Perception in Video-Mediated CommunicationACM Transactions on Computer-Human Interaction10.1145/366034331:3(1-31)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1145/3660343
Show More Cited By

Recommendations

Projected reality: content delivery right onto objects of daily life
ICAT'06: Proceedings of the 16th international conference on Advances in Artificial Reality and Tele-Existence

Spatial augmented reality using controllable projector- camera-systems to project onto objects directly, or Projected Reality as we call it, offers the possibility to augment objects without requiring their users to wear or carry any devices. In order ...
BeHere: a VR/SAR remote collaboration system based on virtual replicas sharing gesture and avatar in a procedural task
Abstract
In this paper, we focus on the significance of remote collaboration using virtual replicas, avatar, and gesture on a procedural task in industry; thus, we present a Virtual Reality (VR)/Spatial Augmented Reality (SAR) remote collaboration system, ...
Beaming: An Asymmetric Telepresence System

The Beaming project recreates, virtually, a real environment; using immersive VR, remote participants can visit the virtual model and interact with the people in the real environment. The real environment doesn't need extensive equipment and can be a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

February 2016

1866 pages

ISBN:9781450335928

DOI:10.1145/2818048

General Chairs:
Darren Gergle
Northwestern University
,
Meredith Ringel Morris
Microsoft Research
,
Program Chairs:
Pernille Bjørn
University of Copenhagen
,
Joseph Konstan
University of Minnesota

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 February 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

CSCW '16

Sponsor:

SIGCHI

CSCW '16: Computer Supported Cooperative Work and Social Computing

February 27 - March 2, 2016

California, San Francisco, USA

Acceptance Rates

CSCW '16 Paper Acceptance Rate 142 of 571 submissions, 25%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

164
Total Citations
View Citations
1,598
Total Downloads

Downloads (Last 12 months)278
Downloads (Last 6 weeks)26

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

IWAI D(2024)Projection mapping technologies: A review of current trends and future directionsProceedings of the Japan Academy, Series B10.2183/pjab.100.012100:3(234-251)Online publication date: 11-Mar-2024
https://doi.org/10.2183/pjab.100.012
Rajaram S(2024)Enabling Safer Augmented Reality Experiences: Usable Privacy Interventions for AR Creators and End-UsersAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686708(1-8)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686708
Schuessler MHormann LDachselt RBlake ARother C(2024)Gazing Heads: Investigating Gaze Perception in Video-Mediated CommunicationACM Transactions on Computer-Human Interaction10.1145/366034331:3(1-31)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1145/3660343
Sidenmark LZhang TAl Lababidi LLi JGrossman T(2024)Desk2Desk: Optimization-based Mixed Reality Workspace Integration for Remote Side-by-side CollaborationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676339(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676339
Rajaram SNuman NKumaravel BMarquardt NWilson A(2024)BlendScape: Enabling End-User Customization of Video-Conferencing Environments through Generative AIProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676326(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676326
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Wong EGrønbæk JVelloso E(2024)The Jamais Vu Effect: Understanding the Fragile Illusion of Co-Presence in Mixed RealityProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661574(2227-2246)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661574
Huang XXiao R(2024)SurfShareProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314187:4(1-24)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631418
Sakashita MThoravi Kumaravel BMarquardt NWilson A(2024)SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642945(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642945
Irlitti ALatifoglu MHoang TSyiem BVetere F(2024)Volumetric Hybrid Workspaces: Interactions with Objects in Remote and Co-located TelepresenceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642814(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642814
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents