research-article

Corsican Twin: Authoring In Situ Augmented Reality Visualisations in Virtual Reality

Authors:

Arnaud Prouzeau,

Wesley Willett,

Tim DwyerAuthors Info & Claims

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

Article No.: 11, Pages 1 - 9

https://doi.org/10.1145/3399715.3399743

Published: 02 October 2020 Publication History

Abstract

We introduce Corsican Twin, a tool for authoring augmented reality data visualisations in virtual reality using digital twins. The system provides users with the contextual information necessary to design embedded and situated data visualisations in a safe and convenient remote setting. We created system via a co-design process which involved people with little or no programming experience. Using the system, we illustrate three potential use cases for situated visualizations in the context of building maintenance, including: (1) on-site equipment debugging and diagnosis; (2) remote incident playback; and (3) operations simulations for future buildings. From feedback gathered during formative evaluations of our prototype tool with domain experts, we discuss implications, opportunities, and challenges for future in situ visualisation design tools.

References

[1]

Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic interaction: designing for a proximity and orientation-aware environment. In ACM International Conference on Interactive Tabletops and Surfaces. ACM, 121--130.

Digital Library

[2]

Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola, and Ivan Poupyrev. 2004. 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, USA.

Digital Library

[3]

Nathalie Bressa, Kendra Wannamaker, Henrik Korsgaard, Wesley Willett, and Jo Vermeulen. 2019. Sketching and Ideation Activities for Situated Visualization Design. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19). ACM, New York, NY, USA, 173--185. https://doi.org/10.1145/3322276.3322326

Digital Library

[4]

Marion Buchenau and Jane Fulton Suri. 2000. Experience Prototyping. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '00). ACM, New York, NY, USA, 424--433. https://doi.org/10.1145/347642.347802

Digital Library

[5]

Colin Burns, Eric Dishman, William Verplank, and Bud Lassiter. 1994. Actors, Hairdos & Videotape---Informance Design. In Conference Companion on Human Factors in Computing Systems (CHI '94). ACM, New York, NY, USA, 119--120. https://doi.org/10.1145/259963.260102

Digital Library

[6]

M. Cavallo and A. G. Forbes. 2019. CAVE-AR: A VR Authoring System to Interactively Design, Simulate, and Debug Multi-user AR Experiences. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 872--873. https://doi.org/10.1109/VR.2019.8798148

[7]

Z. Chen, Y. Su, Y. Wang, Q. Wang, H. Qu, and Y. Wu. 2019. MARVisT: Authoring Glyph-based Visualization in Mobile Augmented Reality. IEEE Transactions on Visualization and Computer Graphics (2019), 1--1. https://doi.org/10.1109/TVCG.2019.2892415

Digital Library

[8]

J. Choi, Y. Kim, M. Lee, G. J. Kim, Y. Nam, and Y. Kwon. 2010. k-MART: Authoring tool for mixed reality contents. In 2010 IEEE International Symposium on Mixed and Augmented Reality. 219--220. https://doi.org/10.1109/ISMAR.2010.5643576

[9]

Sandy Claes, Jorgos Coenen, and Andrew Vande Moere. 2018. Conveying a Civic Issue Through Data via Spatially Distributed Public Visualization and Polling Displays. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (NordiCHI '18). ACM, New York, NY, USA, 597--608. https://doi.org/10.1145/3240167.3240206

Digital Library

[10]

Sandy Claes and Andrew Vande Moere. 2013. Street Infographics: Raising Awareness of Local Issues Through a Situated Urban Visualization. In Proceedings of the 2Nd ACM International Symposium on Pervasive Displays (PerDis '13). ACM, New York, NY, USA, 133--138. https://doi.org/10.1145/2491568.2491597

Digital Library

[11]

M. Cordeil, A. Cunningham, B. Bach, C. Hurter, B. H. Thomas, K. Marriott, and T. Dwyer. 2019. IATK: An Immersive Analytics Toolkit. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 200--209. https://doi.org/10.1109/VR.2019.8797978

[12]

F. M. Dinis, A. S. Guimarães, B. R. Carvalho, and J. P. P. Martins. 2017. Virtual and augmented reality game-based applications to civil engineering education. In 2017 IEEE Global Engineering Education Conference (EDUCON). 1683--1688. https://doi.org/10.1109/EDUCON.2017.7943075

[13]

Alexandre Dumas. 1844. The Corsican Brothers.

[14]

N. ElSayed, B. Thomas, K. Marriott, J. Piantadosi, and R. Smith. 2015. Situated Analytics. In 2015 Big Data Visual Analytics (BDVA). 1--8. https://doi.org/10.1109/BDVA.2015.7314302

[15]

Barrett Ens and Pourang Irani. 2016. Spatial analytic interfaces: Spatial user interfaces for in situ visual analytics. IEEE computer graphics and applications 37, 2 (2016), 66--79.

[16]

Eva Eriksson, Martin Ludvigsen, A Lykke-Olesen, and R Nielsen. 2006. Bthere or be Square: A Method for Extreme Contextualization of Design. (01 2006).

[17]

George W Fitzmaurice. 1993. Situated information spaces and spatially aware palmtop computers. Commun. ACM 36, 7 (1993), 38--50.

Digital Library

[18]

Mani Golparvar-Fard, Feniosky Peña-Mora, and Silvio Savarese. 2009. D4AR-a 4-dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication. Journal of information technology in construction 14, 13 (2009), 129--153.

[19]

Nicolai Brodersen Hansen and Peter Dalsgaard. 2012. The Productive Role of Material Design Artefacts in Participatory Design Events. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12). ACM, New York, NY, USA, 665--674. https://doi.org/10.1145/2399016.2399117

Digital Library

[20]

S. Henderson and S. Feiner. 2011. Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Transactions on Visualization and Computer Graphics 17, 10 (Oct 2011), 1355--1368. https://doi.org/10.1109/TVCG.2010.245

Digital Library

[21]

Valentin Heun, James Hobin, and Pattie Maes. 2013. Reality editor: programming smarter objects. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. ACM, 307--310.

Digital Library

[22]

Ken Hinckley, Randy Pausch, Dennis Proffitt, James Patten, and Neal Kassell. 1997. Cooperative bimanual action. In CHI, Vol. 97. 27--34.

[23]

Henrik Korsgaard, Nicolai Brodersen Hansen, Ditte Basballe, Peter Dalsgaard, and Kim Halskov. 2012. Odenplan: A Media FaÇAde Design Process. In Proceedings of the 4th Media Architecture Biennale Conference: Participation (MAB '12). ACM, New York, NY, USA, 23--32. https://doi.org/10.1145/2421076.2421081

Digital Library

[24]

Gun A. Lee, Gerard J. Kim, and Mark Billinghurst. 2005. Immersive Authoring: What You eXperience Is What You Get (WYXIWYG). Commun. ACM 48, 7 (July 2005), 76--81. https://doi.org/10.1145/1070838.1070840

Digital Library

[25]

B. MacIntyre, M. Gandy, J. Bolter, S. Dow, and B. Hannigan. 2003. DART: the Designer's Augmented Reality Toolkit. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. 329--330. https://doi.org/10.1109/ISMAR.2003.1240744

[26]

Matterport. 2019. Matterport. https://matterport.com/

[27]

Microsoft. 2019. Microsoft Hololens. https://www.microsoft.com/en-us/hololens

[28]

Microsoft. 2019. Microsoft Layout. https://dynamics.microsoft.com/en-au/mixed-reality/layout/

[29]

William Odom, John Zimmerman, Scott Davidoff, Jodi Forlizzi, Anind K. Dey, and Min Kyung Lee. 2012. A Fieldwork of the Future with User Enactments. In Proceedings of the Designing Interactive Systems Conference (DIS '12). ACM, New York, NY, USA, 338--347. https://doi.org/10.1145/2317956.2318008

Digital Library

[30]

Antti Oulasvirta, Esko Kurvinen, and Tomi Kankainen. 2003. Understanding contexts by being there: case studies in bodystorming. Personal and Ubiquitous Computing 7, 2 (01 Jul 2003), 125--134. https://doi.org/10.1007/s00779-003-0238-7

Digital Library

[31]

R. Sicat, J. Li, J. Choi, M. Cordeil, W. Jeong, B. Bach, and H. Pfister. 2019. DXR: A Toolkit for Building Immersive Data Visualizations. IEEE Transactions on Visualization and Computer Graphics 25, 1 (Jan 2019), 715--725. https://doi.org/10.1109/TVCG.2018.2865152

Digital Library

[32]

Richard Stoakley, Matthew Conway, and Y Pausch. 1995. Virtual Reality on a WIM: Interactive Worlds in Miniature. (02 1995). https://doi.org/10.1145/223904.223938

Digital Library

[33]

Bruce H Thomas, Gregory F Welch, Pierre Dragicevic, Niklas Elmqvist, Pourang Irani, Yvonne Jansen, Dieter Schmalstieg, Aurélien Tabard, Neven AM ElSayed, Ross T Smith, et al. 2018. Situated Analytics.

[34]

Fernando Vera, J. Alfredo Sánchez, and Ofelia Cervantes. 2017. A Platform for Creating Augmented Reality Content by End Users. In Applications for Future Internet, Enrique Sucar, Oscar Mayora, and Enrique Munoz de Cote (Eds.). Springer International Publishing, Cham, 167--171.

[35]

J. Vermeulen, F. Kawsar, A. L. Simeone, G. Kortuem, K. Luyten, and K. Coninx. 2012. Informing the design of situated glyphs for a care facility. In 2012 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 89--96. https://doi.org/10.1109/VLHCC.2012.6344490

[36]

Rebekka Volk, Julian Stengel, and Frank Schultmann. 2014. Building Information Modeling (BIM) for existing buildings --- Literature review and future needs. Automation in Construction 38 (2014), 109--127. https://doi.org/10.1016/j.autcon.2013.10.023

[37]

Yuan Xing Wang, Tobias Langlotz, Mark Billinghurst, and Tobin Bell. 2009. An Authoring Tool for Mobile Phone AR Environments.

[38]

W. Willett, Y. Jansen, and P. Dragicevic. 2017. Embedded Data Representations. IEEE Transactions on Visualization and Computer Graphics 23, 1 (Jan 2017), 461--470. https://doi.org/10.1109/TVCG.2016.2598608

Digital Library

[39]

Mike Wozniewski and Paul Warne. 2011. Towards in situ authoring of augmented reality content. In ISMAR.

Cited By

Adigun GAjani YEnakrire R(2024)The Intelligent Libraries: Innovation for a Sustainable Knowledge System in the Fifth (5th) Industrial RevolutionLibri10.1515/libri-2023-0111Online publication date: 27-Jun-2024
https://doi.org/10.1515/libri-2023-0111
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/3648613Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1145/3648613
Elsayed NMarriott KSmith RThomas B(2024)Situated Analytics Process and MantraExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650814(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650814
Show More Cited By

Index Terms

Corsican Twin: Authoring In Situ Augmented Reality Visualisations in Virtual Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
  2. Visualization
    1. Visualization techniques

Recommendations

Tangible Globes for Data Visualisation in Augmented Reality
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Head-mounted augmented reality (AR) displays allow for the seamless integration of virtual visualisation with contextual tangible references, such as physical (tangible) globes. We explore the design of immersive geospatial data visualisation with AR and ...
A Pilot Study Comparing User Interactions Between Augmented and Virtual Reality
Advances in Visual Computing
Abstract
Immersive Analytics (IA) and consumer adoption of augmented reality (AR) and virtual reality (VR) head-mounted displays (HMDs) are both rapidly growing. When used in conjunction, stereoscopic IA environments can offer improved user understanding ...
I Did Not Notice: A Comparison of Immersive Analytics with Augmented and Virtual Reality
CHI EA '24: Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems
Immersive environments enable users to engage in embodied interaction, enhancing the sensemaking processes involved in completing tasks such as immersive analytics. Previous comparative studies on immersive analytics using augmented and virtual realities ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

September 2020

613 pages

ISBN:9781450375351

DOI:10.1145/3399715

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AVI '20

AVI '20: International Conference on Advanced Visual Interfaces

September 28 - October 2, 2020

Salerno, Italy

Acceptance Rates

AVI '20 Paper Acceptance Rate 36 of 123 submissions, 29%;

Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
779
Total Downloads

Downloads (Last 12 months)213
Downloads (Last 6 weeks)19

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Adigun GAjani YEnakrire R(2024)The Intelligent Libraries: Innovation for a Sustainable Knowledge System in the Fifth (5th) Industrial RevolutionLibri10.1515/libri-2023-0111Online publication date: 27-Jun-2024
https://doi.org/10.1515/libri-2023-0111
Perelman GSerrano MDubois E(2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/3648613Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1145/3648613
Elsayed NMarriott KSmith RThomas B(2024)Situated Analytics Process and MantraExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650814(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650814
Cho HYan YTodi KParent MSmith MJonker TBenko HLindlbauer D(2024)MineXR: Mining Personalized Extended Reality InterfacesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642394(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642394
Zhu QWang ZZeng WTong WLin WMa X(2024)Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public EnvironmentsProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642049(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642049
Friedl‐Knirsch JPointecker FPfistermüller SStach CAnthes CRoth D(2024)A Systematic Literature Review of User Evaluation in Immersive AnalyticsComputer Graphics Forum10.1111/cgf.1511143:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15111
Shin SBatch AButcher PRitsos PElmqvist N(2024)The Reality of the Situation: A Survey of Situated AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328554630:8(5147-5164)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3285546
Simonson AShao G(2024)An Authoring Tool for Mixed Reality Interfaces for Digital Twins in ManufacturingAdvances in Production Management Systems. Production Management Systems for Volatile, Uncertain, Complex, and Ambiguous Environments10.1007/978-3-031-71633-1_27(380-393)Online publication date: 7-Sep-2024
https://doi.org/10.1007/978-3-031-71633-1_27
Soman RNikolić DSanchez B(2024)Extended Reality as a Catalyst for Circular Economy Transition in the Built EnvironmentA Circular Built Environment in the Digital Age10.1007/978-3-031-39675-5_10(171-193)Online publication date: 4-Jan-2024
https://doi.org/10.1007/978-3-031-39675-5_10
Chauvergne EHachet MProuzeau A(2023)Authoring Interactive and Immersive Experiences Using Programming by DemonstrationProceedings of the 34th Conference on l'Interaction Humain-Machine10.1145/3583961.3583981(1-13)Online publication date: 3-Apr-2023
https://dl.acm.org/doi/10.1145/3583961.3583981
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents