research-article

Collaborating Across Realities: Analytical Lenses for Understanding Dyadic Collaboration in Transitional Interfaces

Authors:

Jan-Henrik Schröder,

Daniel Schacht,

Anita Marie Hamurculu,

Hans-Christian JetterAuthors Info & Claims

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 97, Pages 1 - 16

https://doi.org/10.1145/3544548.3580879

Published: 19 April 2023 Publication History

Abstract

Transitional Interfaces are a yet underexplored, emerging class of cross-reality user interfaces that enable users to freely move along the reality-virtuality continuum during collaboration. To analyze and understand how such collaboration unfolds, we propose four analytical lenses derived from an exploratory study of transitional collaboration with 15 dyads. While solving a complex spatial optimization task, participants could freely switch between three contexts, each with different displays (desktop screens, tablet-based augmented reality, head-mounted virtual reality), input techniques (mouse, touch, handheld controllers), and visual representations (monoscopic and allocentric 2D/3D maps, stereoscopic egocentric views). Using the rich qualitative and quantitative data from our study, we evaluated participants’ perceptions of transitional collaboration and identified commonalities and differences between dyads. We then derived four lenses including metrics and visualizations to analyze key aspects of transitional collaboration: (1) place and distance, (2) temporal patterns, (3) group use of contexts, (4) individual use of contexts.

Supplementary Material

MP4 File (3544548.3580879-video-figure.mp4)

Video Figure

Download
54.94 MB

MP4 File (3544548.3580879-video-preview.mp4)

Video Preview

Download
8.64 MB

MP4 File (3544548.3580879-talk-video.mp4)

Pre-recorded Video Presentation

Download
33.71 MB

References

[1]

Huidong Bai, Prasanth Sasikumar, Jing Yang, and Mark Billinghurst. 2020. A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376550

Digital Library

[2]

Mark Billinghurst, Hirokazu Kato, and Ivan Poupyrev. 2001. The magic book: A transitional AR interface. Computers & Graphics 25, 5 (2001), 745–753. https://doi.org/10.1016/S0097-8493(01)00117-0

[3]

Frederik Brudy, Joshua Kevin Budiman, Steven Houben, and Nicolai Marquardt. 2018. Investigating the Role of an Overview Device in Multi-Device Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173874

Digital Library

[4]

Felipe G. Carvalho, Daniela G. Trevisan, and Alberto Raposo. 2012. Toward the design of transitional interfaces: an exploratory study on a semi-immersive hybrid user interface. Virtual Reality 16, 4 (Nov. 2012), 271–288. https://doi.org/10.1007/s10055-011-0205-y

Digital Library

[5]

Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies 131 (Nov. 2019), 81–98. https://doi.org/10.1016/j.ijhcs.2019.05.011

Digital Library

[6]

Thomas Franke, Christiane Attig, and Daniel Wessel. 2019. A Personal Resource for Technology Interaction: Development and Validation of the Affinity for Technology Interaction (ATI) Scale. International Journal of Human–Computer Interaction 35, 6 (April 2019), 456–467. https://doi.org/10.1080/10447318.2018.1456150

[7]

Bernhard Fröhler, C. Anthes, Christoph, Fabian Pointecker, Judith Friedl, D. Schwajda, A. Riegler, Andreas, Shailesh Tripathi, Clemens Holzmann, Manuel Brunner, Herbert Jodlbauer, Hans-Christian Jetter, and Christoph Heinzl. 2022. A Survey on Cross‐Virtuality Analytics. Computer Graphics Forum(2022), 465–494. https://doi.org/10.1111/cgf.14447

[8]

Alexander Gall, Bernhard Fröhler, Julia Maurer, Johann Kastner, and Christoph Heinzl. 2022. Cross-virtuality analysis of rich X-ray computed tomography data for materials science applications. Nondestructive Testing and Evaluation (May 2022), 1–16. https://doi.org/10.1080/10589759.2022.2075864

[9]

Jeronimo Gustavo Grandi, Henrique Galvan Debarba, and Anderson Maciel. 2019. Characterizing Asymmetric Collaborative Interactions in Virtual and Augmented Realities. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Osaka, Japan, 127–135. https://doi.org/10.1109/VR.2019.8798080

[10]

Raphael Grasset, Andreas Dunser, and Mark Billinghurst. 2008. Moving Between Contexts - A User Evaluation of a Transitional Interface. In 18th International Conference on Artificial Reality and Telexistence (ICAT 2008). Yokohama Japan, 137–143.

[11]

Raphael Grasset, Philip Lamb, and Mark Billinghurst. 2005. Evaluation of Mixed-Space Collaboration. In Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality(ISMAR ’05). IEEE Computer Society, USA, 90–99. https://doi.org/10.1109/ISMAR.2005.30

Digital Library

[12]

Raphael Grasset, Julian Looser, and Mark Billinghurst. 2006. Transitional interface: concept, issues and framework. In Proceedings of the 5th IEEE/ACM International Symposium on Mixed and Augmented Reality(ISMAR ’06). IEEE Computer Society, USA, 231–232. https://doi.org/10.1109/ismar.2006.297819

Digital Library

[13]

Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 4021–4033. https://doi.org/10.1145/3025453.3025683

Digital Library

[14]

Carl Gutwin and Saul Greenberg. 2002. A Descriptive Framework of Workspace Awareness for Real-Time Groupware. Computer Supported Cooperative Work (CSCW) 11, 3-4 (Sept. 2002), 411–446. https://doi.org/10.1023/A:1021271517844

Digital Library

[15]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology. Vol. 52. Elsevier, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

[16]

Petra Isenberg, Danyel Fisher, Sharoda A. Paul, Meredith R. Morris, Kori Inkpen, and Mary Czerwinski. 2012. Co-Located Collaborative Visual Analytics around a Tabletop Display. IEEE Transactions on Visualization and Computer Graphics 18, 5 (May 2012), 689–702. https://doi.org/10.1109/TVCG.2011.287

Digital Library

[17]

Hans-Christian Jetter, Jan-Henrik Schröder, Jan Gugenheimer, Mark Billinghurst, Christoph Anthes, Mohamed Khamis, and Tiare Feuchtner. 2021. Transitional Interfaces in Mixed and Cross-Reality: A new frontier?. In Interactive Surfaces and Spaces(ISS ’21). Association for Computing Machinery, New York, NY, USA, 46–49. https://doi.org/10.1145/3447932.3487940

Digital Library

[18]

Robert Johansen. 1988. Groupware: computer support for business teams. Free Press ; Collier Macmillan, New York : London.

Digital Library

[19]

S. Kullback and R. A. Leibler. 1951. On Information and Sufficiency. The Annals of Mathematical Statistics 22, 1 (March 1951), 79–86. https://doi.org/10.1214/aoms/1177729694

[20]

Sebastian Linxen, Christian Sturm, Florian Brühlmann, Vincent Cassau, Klaus Opwis, and Katharina Reinecke. 2021. How WEIRD is CHI?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–14. https://doi.org/10.1145/3411764.3445488

Digital Library

[21]

Nicolai Marquardt, Ken Hinckley, and Saul Greenberg. 2012. Cross-device interaction via micro-mobility and f-formations. In Proceedings of the 25th annual ACM symposium on User interface software and technology - UIST ’12. ACM Press, Cambridge, Massachusetts, USA, 13–22. https://doi.org/10.1145/2380116.2380121

Digital Library

[22]

Frank Maurer, Craig Anslow, Joaquim Jorge, and Mauricio Sousa. 2022. Enhancing cross-reality applications and user experiences. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces(AVI 2022). Association for Computing Machinery, New York, NY, USA, 1–3. https://doi.org/10.1145/3531073.3535256

Digital Library

[23]

Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77, 12 (1994), 1321–1329. Publisher: The Institute of Electronics, Information and Communication Engineers.

[24]

Thomas Neumayr, Hans-Christian Jetter, Mirjam Augstein, Judith Friedl, and Thomas Luger. 2018. Domino: A Descriptive Framework for Hybrid Collaboration and Coupling Styles in Partially Distributed Teams. Proc. ACM Hum.-Comput. Interact. 2, CSCW (Nov. 2018). https://doi.org/10.1145/3274397

Digital Library

[25]

Thammathip Piumsomboon, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. CoVAR: A Collaborative Virtual and Augmented Reality System for Remote Collaboration. In SIGGRAPH Asia 2017 Emerging Technologies(SA ’17). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3132818.3132822

Digital Library

[26]

Thomas Plank, Hans-Christian Jetter, Roman Rädle, Clemens N. Klokmose, Thomas Luger, and Harald Reiterer. 2017. Is Two Enough?!: Studying Benefits, Barriers, and Biases of Multi-Tablet Use for Collaborative Visualization. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17. ACM Press, Denver, Colorado, USA, 4548–4560. https://doi.org/10.1145/3025453.3025537

Digital Library

[27]

Fabian Pointecker, Judith Friedl, Daniel Schwajda, Hans-Christian Jetter, and Christoph Anthes. 2022. Bridging the Gap Across Realities: Visual Transitions Between Virtual and Augmented Reality. In 2022 IEEE Interational Symposium on Mixed and Augmented Reality (ISMAR). Singapore. (to appear).

[28]

Fabian Pointecker, Hans-Christian Jetter, and Christoph Anthes. 2020. Exploration of Visual Transitions Between Virtual and Augmented Reality. In 4th Workshop on Immersive Analytics: Envisioning Future Productivity for Immersive Analytics // @CHI 2020 Honolulu. https://drive.google.com/file/d/1V1lXIk-DPjcfuJg2LgaS_UfYXQq-7e2k/view

[29]

Patrick Reipschläger and Raimund Dachselt. 2019. DesignAR: Immersive 3D-Modeling Combining Augmented Reality with Interactive Displays. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces(ISS ’19). ACM, New York, NY, USA, 29–42. https://doi.org/10.1145/3343055.3359718

Digital Library

[30]

Patrick Reipschläger, Severin Engert, and Raimund Dachselt. 2020. Augmented Displays: Seamlessly Extending Interactive Surfaces with Head-Mounted Augmented Reality. In Proceedings of the 2020 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3334480.3383138

Digital Library

[31]

Patrick Reipschläger, Tamara Flemisch, and Raimund Dachselt. 2021. Personal Augmented Reality for Information Visualization on Large Interactive Displays. IEEE Transactions on Visualization and Computer Graphics 27, 2 (Feb. 2021), 1182–1192. https://doi.org/10.1109/TVCG.2020.3030460

[32]

Andreas Riegler, Christoph Anthes, Hans-Christian Jetter, Christoph Heinzl, Clemens Holzmann, Jodlbauer Herbert, Manuel Brunner, Stefan Auer, Judith Friedl, and Bernhard Fröhler. 2020. Cross-Virtuality Visualization, Interaction and Collaboration. In International Workshop on Cross-Reality (XR) Interaction co-located with 14th ACM International Conference on Interactive Surfaces and Spaces. http://ceur-ws.org/Vol-2779/paper1.pdf

[33]

Joan Sol Roo and Martin Hachet. 2017. One Reality: Augmenting How the Physical World is Experienced by combining Multiple Mixed Reality Modalities. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, Québec City QC Canada, 787–795. https://doi.org/10.1145/3126594.3126638

Digital Library

[34]

Stacey D. Scott, Carpendale Sheelagh, and Kori M. Inkpen. 2004. Territoriality in collaborative tabletop workspaces. In Proceedings of the 2004 ACM conference on Computer supported cooperative work - CSCW ’04. ACM Press, Chicago, Illinois, USA, 294. https://doi.org/10.1145/1031607.1031655

Digital Library

[35]

James Sellers, William S. Helton, Katharina Näswall, Gregory J. Funke, and Benjamin A. Knott. 2014. Development of the Team Workload Questionnaire (TWLQ). Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58, 1 (Sept. 2014), 989–993. https://doi.org/10.1177/1541931214581207

[36]

Adalberto L. Simeone, Mohamed Khamis, Augusto Esteves, Florian Daiber, Matjaž Kljun, Klen Čopič Pucihar, Poika Isokoski, and Jan Gugenheimer. 2020. International Workshop on Cross-Reality (XR) Interaction. In Companion Proceedings of the 2020 Conference on Interactive Surfaces and Spaces(ISS ’20). Association for Computing Machinery, New York, NY, USA, 111–114. https://doi.org/10.1145/3380867.3424551

Digital Library

[37]

Anthony Tang, Melanie Tory, Barry Po, Petra Neumann, and Sheelagh Carpendale. 2006. Collaborative Coupling over Tabletop Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’06). Association for Computing Machinery, New York, NY, USA, 1181–1190. https://doi.org/10.1145/1124772.1124950

Digital Library

[38]

Philip Tuddenham and Peter Robinson. 2009. Territorial Coordination and Workspace Awareness in Remote Tabletop Collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’09). Association for Computing Machinery, New York, NY, USA, 2139–2148. https://doi.org/10.1145/1518701.1519026

Digital Library

[39]

Edward R. Tufte. 2001. The visual display of quantitative information (2nd ed.). Graphics Press, Cheshire, Conn.

Cited By

You JWu ZWei WLi NYang Y(2024)Evolution of Industrial Robots from the Perspective of the Metaverse: Integration of Virtual and Physical Realities and Human–Robot CollaborationApplied Sciences10.3390/app1414636914:14(6369)Online publication date: 22-Jul-2024
https://doi.org/10.3390/app14146369
Schmidt LYigitbas E(2024)Development and Usability Evaluation of Transitional Cross-Reality InterfacesProceedings of the ACM on Human-Computer Interaction10.1145/36646378:EICS(1-32)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3664637
Stacchio LVallasciani GAugello GCarrador SCascarano PMarfia G(2024)WiXaRd: Towards a Holistic Distributed Platform for Multi-Party and Cross-Reality WebXR Experiences2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00052(264-272)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00052
Show More Cited By

Index Terms

Collaborating Across Realities: Analytical Lenses for Understanding Dyadic Collaboration in Transitional Interfaces
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Computer supported cooperative work
  2. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

In-Vehicle Transitional Interfaces: User Journey in Virtual Reality
AutomotiveUI '22: Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

In this poster, we investigate novel user-centered methods and techniques for cross-virtuality user journeys for designing in-vehicle experiences. We argue that rapid prototyping of automotive human-machine interfaces (HMIs) can be achieved with mixed ...
Towards Intelligent Interfaces for Mixed-Focus Collaboration
UMAP '18: Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization

Large, high-resolution displays are highly suitable for creation of digital environments for co-located collaborative task solving. Yet, placing multiple users in a shared environment may increase the risk of interferences, thus causing mental ...
Transitional Interfaces in Mixed and Cross-Reality: A new frontier?
ISS '21: Companion Proceedings of the 2021 Conference on Interactive Surfaces and Spaces

Transitional interfaces (TIs) and related concepts such as cross-reality (XR) or cross-virtuality (XV) are key topics for future HCI and AR/VR research. Future TIs will enable users to freely move between different locations within the reality-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

14911 pages

ISBN:9781450394215

DOI:10.1145/3544548

Editors:
Albrecht Schmidt
LMU Munich, Germany60028717
,
Kaisa Väänänen
Tampere University, Finland60011170
,
Tesh Goyal
Google Research, USA60006191
,
Per Ola Kristensson
University of Cambridge, UK60031101
,
Anicia Peters
University of Namibia, Namibia60072704
,
Stefanie Mueller
Massachusetts Institute of Technology, USA60022195
,
Julie R. Williamson
University of Glasgow, UK60001490
,
Max L. Wilson
University of Nottingham, UK60015138

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Government of Upper Austria

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
1,931
Total Downloads

Downloads (Last 12 months)1,393
Downloads (Last 6 weeks)134

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

You JWu ZWei WLi NYang Y(2024)Evolution of Industrial Robots from the Perspective of the Metaverse: Integration of Virtual and Physical Realities and Human–Robot CollaborationApplied Sciences10.3390/app1414636914:14(6369)Online publication date: 22-Jul-2024
https://doi.org/10.3390/app14146369
Schmidt LYigitbas E(2024)Development and Usability Evaluation of Transitional Cross-Reality InterfacesProceedings of the ACM on Human-Computer Interaction10.1145/36646378:EICS(1-32)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3664637
Stacchio LVallasciani GAugello GCarrador SCascarano PMarfia G(2024)WiXaRd: Towards a Holistic Distributed Platform for Multi-Party and Cross-Reality WebXR Experiences2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00052(264-272)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00052
Cho HYuan BHart JChang ZCao JChang EBillinghurst M(2023)Time Travellers: An Asynchronous Cross Reality Collaborative System2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00186(848-853)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00186
Lugtenberg GMori KMatoba YTeo TBillinghurst M(2023)The MagicBook Revisited2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00177(801-806)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00177
Castillo JAssadian SBatmaz A(2023)Where We Stand and Where to Go: Building Bridges Between Real and Virtual Worlds for Collaboration2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00053(228-233)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00053
Schröder JJetter H(2023)Towards a Model for Space and Time in Transitional Collaboration2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00052(223-227)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00052
Grønbæk J(2023)Transitional Blended Realities: Linking between Video and Mixed Reality for Distributed Team Collaboration2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00051(220-222)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00051
Zagermann JHubenschmid SFink DWieland JReiterer HFeuchtner T(2023)Challenges and Opportunities for Collaborative Immersive Analytics with Hybrid User Interfaces2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00044(191-195)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00044
Kern FLatoschik M(2023)Reality Stack I/O: A Versatile and Modular Framework for Simplifying and Unifying XR Applications and Research2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00023(74-76)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00023
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents