research-article

Open access

STREAM: Exploring the Combination of Spatially-Aware Tablets with Augmented Reality Head-Mounted Displays for Immersive Analytics

Authors:

Sebastian Hubenschmid,

Johannes Zagermann,

Simon Butscher, and

Harald ReitererAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

Article No.: 469, Pages 1 - 14

https://doi.org/10.1145/3411764.3445298

Published: 07 May 2021 Publication History

All formats PDF

Abstract

Recent research in the area of immersive analytics demonstrated the utility of head-mounted augmented reality devices for visual data analysis. However, it can be challenging to use the by default supported mid-air gestures to interact with visualizations in augmented reality (e.g. due to limited precision). Touch-based interaction (e.g. via mobile devices) can compensate for these drawbacks, but is limited to two-dimensional input. In this work we present STREAM: Spatially-aware Tablets combined with Augmented Reality Head-Mounted Displays for the multimodal interaction with 3D visualizations. We developed a novel eyes-free interaction concept for the seamless transition between the tablet and the augmented reality environment. A user study reveals that participants appreciated the novel interaction concept, indicating the potential for spatially-aware tablets in augmented reality. Based on our findings, we provide design insights to foster the application of spatially-aware touch devices in augmented reality and research implications indicating areas that need further investigation.

Supplementary Material

MP4 File (3411764.3445298_videofigure.mp4)

Supplemental video

Download
96.37 MB

MP4 File (3411764.3445298_videopreview.mp4)

Preview video

Download
17.84 MB

References

[1]

Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, Jürgen Steimle, and Tino Weinkauf. 2015. Performance and Ergonomics of Touch Surfaces: A Comparative Study Using Biomechanical Simulation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15. ACM Press, Seoul, Republic of Korea, 1817–1826. https://doi.org/10.1145/2702123.2702607

Digital Library

[2]

Sriram Karthik Badam, Fereshteh Amini, Niklas Elmqvist, and Pourang Irani. 2016. Supporting Visual Exploration for Multiple Users in Large Display Environments, In 2016 IEEE Conference on Visual Analytics Science and Technology (VAST). 2016 IEEE Conference on Visual Analytics Science and Technology, VAST 2016 - Proceedings, 1–10. https://doi.org/10.1109/VAST.2016.7883506

[3]

Huidong Bai, Lei Gao, and Mark Billinghurst. 2017. 6DoF Input for Hololens Using Vive Controller. In SIGGRAPH Asia 2017 Mobile Graphics & Interactive Applications on - SA ’17. ACM Press, Bangkok, Thailand, 1–1. https://doi.org/10.1145/3132787.3132814

Digital Library

[4]

Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment, In ACM International Conference on Interactive Tabletops and Surfaces - ITS '10. ACM International Conference on Interactive Tabletops and Surfaces - ITS ’10, 121. https://doi.org/10.1145/1936652.1936676

Digital Library

[5]

Andrea Batch, Andrew Cunningham, Maxime Cordeil, Niklas Elmqvist, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2019. There Is No Spoon: Evaluating Performance, Space Use, and Presence with Expert Domain Users in Immersive Analytics. IEEE Transactions on Visualization and Computer Graphics 26, 1 (jan 2019), 1–1. https://doi.org/10.31219/osf.io/wzqbu

[6]

Jens Bauer and Achim Ebert. 2015. Virtual Buttons for Eyes-Free Interaction: A Study. In Human-Computer Interaction – INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque, and Marco Winckler (Eds.). Vol. 9296. Springer International Publishing, Cham, 47–54. https://doi.org/10.1007/978-3-319-22701-6_4

Digital Library

[7]

Jens Bauer, Achim Ebert, Oliver Kreylos, and Bernd Hamann. 2013. Marking Menus for Eyes-Free Interaction Using Smart Phones and Tablets. In Availability, Reliability, and Security in Information Systems and HCI, Alfredo Cuzzocrea, Christian Kittl, Dimitris E. Simos, Edgar Weippl, and Lida Xu (Eds.). Vol. 8127. Springer Berlin Heidelberg, Berlin, Heidelberg, 481–494. https://doi.org/10.1007/978-3-642-40511-2_35

[8]

Michel Beaudouin-Lafon. 2000. Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’00). Association for Computing Machinery, New York, NY, USA, 446–453. https://doi.org/10.1145/332040.332473

Digital Library

[9]

Hrvoje Benko and Steven Feiner. 2007. Balloon Selection: A Multi-Finger Technique for Accurate Low-Fatigue 3D Selection. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI ’07). IEEE Computer Society, Washington, DC, USA, 79–86. https://doi.org/10.1109/3DUI.2007.340778

[10]

Lonni Besançon, Mickael Sereno, Lingyun Yu, Mehdi Ammi, and Tobias Isenberg. 2019. Hybrid Touch/Tangible Spatial 3D Data Selection. Computer Graphics Forum 38, 3 (jun 2019), 553–567. https://doi.org/10.1111/cgf.13710

[11]

Verena Biener, Daniel Schneider, Travis Gesslein, Alexander Otte, Bastian Kuth, Per Ola Kristensson, Eyal Ofek, Michel Pahud, and Jens Grubert. 2020. Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers. IEEE Transactions on Visualization and Computer Graphics 26, 12 (dec 2020), 3490–3502. https://doi.org/10.1109/tvcg.2020.3023567

[12]

Sebastian Boring, Marko Jurmu, and Andreas Butz. 2009. Scroll, Tilt or Move It: Using Mobile Phones to Continuously Control Pointers on Large Public Displays. In Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7(OZCHI ’09). Association for Computing Machinery, New York, NY, USA, 161–168. https://doi.org/10.1145/1738826.1738853

Digital Library

[13]

Stephen Brewster, Joanna Lumsden, Marek Bell, Malcolm Hall, and Stuart Tasker. 2003. Multimodal ’eyes-Free’ Interaction Techniques for Wearable Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’03). Association for Computing Machinery, New York, NY, USA, 473–480. https://doi.org/10.1145/642611.642694

Digital Library

[14]

Gerd Bruder, Frank Steinicke, and Wolfgang Stürzlinger. 2013. Effects of Visual Conflicts on 3D Selection Task Performance in Stereoscopic Display Environments. In 2013 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 115–118. https://doi.org/10.1109/3dui.2013.6550207

[15]

Wolfgang Büschel, Annett Mitschick, Thomas Meyer, and Raimund Dachselt. 2019. Investigating Smartphone-Based Pan and Zoom in 3D Data Spaces in Augmented Reality. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI ’19. ACM Press, Taipei, Taiwan, 1–13. https://doi.org/10.1145/3338286.3340113

Digital Library

[16]

Wolfgang Büschel, Patrick Reipschläger, Ricardo Langner, and Raimund Dachselt. 2017. Investigating the Use of Spatial Interaction for 3D Data Visualization on Mobile Devices. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces(ISS ’17). Association for Computing Machinery, Brighton, United Kingdom, 62–71. https://doi.org/10.1145/3132272.3134125

Digital Library

[17]

Peter W. S. Butcher, Nigel W. John, and Panagiotis D. Ritsos. 2019. VRIA - A Framework for Immersive Analytics on the Web. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems(CHI EA ’19). Association for Computing Machinery, Glasgow, Scotland Uk, 1–6. https://doi.org/10.1145/3290607.3312798

Digital Library

[18]

Simon Butscher, Sebastian Hubenschmid, Jens Müller, Johannes Fuchs, and Harald Reiterer. 2018. Clusters, Trends, and Outliers: How Immersive Technologies Can Facilitate the Collaborative Analysis of Multidimensional Data. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, New York, New York, USA, 1–12. https://doi.org/10.1145/3173574.3173664

Digital Library

[19]

Marco Cavallo, Mishal Dolakia, Matous Havlena, Kenneth Ocheltree, and Mark Podlaseck. 2019. Immersive Insights: A Hybrid Analytics System for Collaborative Exploratory Data Analysis. In 25th ACM Symposium on Virtual Reality Software and Technology on - VRST ’19. ACM Press, Parramatta, NSW, Australia, 1–12. https://doi.org/10.1145/3359996.3364242

Digital Library

[20]

Li-Wei Chan, Hui-Shan Kao, Mike Y. Chen, Ming-Sui Lee, Jane Hsu, and Yi-Ping Hung. 2010. Touching the Void: Direct-Touch Interaction for Intangible Displays, In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2625–2634. https://doi.org/10.1145/1753326.1753725

Digital Library

[21]

Maxime Cordeil, Andrew Cunningham, Benjamin Bach, Christophe Hurter, Bruce H. Thomas, Kim Marriott, and Tim Dwyer. 2019. IATK: An Immersive Analytics Toolkit. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Osaka, Japan, 200–209. https://doi.org/10.1109/VR.2019.8797978

[22]

Maxime Cordeil, Andrew Cunningham, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2017. ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation. In Proceedings of the ACM Symposium on User Interface and Software Technology (UIST ’17). ACM Press, New York, NY, USA, 71–83. https://doi.org/10.1145/3126594.3126613

Digital Library

[23]

Tobias Drey, Jan Gugenheimer, Julian Karlbauer, Maximilian Milo, and Enrico Rukzio. 2020. VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, Honolulu, HI, USA, 1–14. https://doi.org/10.1145/3313831.3376628

Digital Library

[24]

Niklas Elmqvist, Andrew Vande Moere, Hans-Christian Jetter, Daniel Cernea, Harald Reiterer, and T J Jankun-Kelly. 2011. Fluid Interaction for Information Visualization. Information Visualization - Special Issue on State of the Field and New Research Directions 10, 4 (2011), 327–340. https://doi.org/10.1177/1473871611413180

Digital Library

[25]

Steven Feiner and Ari Shamash. 1991. Hybrid User Interfaces: Breeding Virtually Bigger Interfaces for Physically Smaller Computers. In Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology(UIST ’91). Association for Computing Machinery, Hilton Head, South Carolina, USA, 9–17. https://doi.org/10.1145/120782.120783

Digital Library

[26]

Jens Grubert, Matthias Heinisch, Aaron Quigley, and Dieter Schmalstieg. 2015. MultiFi: Multi Fidelity Interaction with Displays On and Around the Body. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). Association for Computing Machinery, Seoul, Republic of Korea, 3933–3942. https://doi.org/10.1145/2702123.2702331

Digital Library

[27]

Jens Grubert, Matthias Kranz, and Aaron Quigley. 2016. Challenges in Mobile Multi-Device Ecosystems. mUX: The Journal of Mobile User Experience 5, 1 (aug 2016). https://doi.org/10.1186/s13678-016-0007-y

[28]

Mark Hancock, Otmar Hilliges, Christopher Collins, Dominikus Baur, and Sheelagh Carpendale. 2009. Exploring Tangible and Direct Touch Interfaces for Manipulating 2D and 3D Information on a Digital Table. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS ’09. ACM Press, Banff, Alberta, Canada, 77. https://doi.org/10.1145/1731903.1731921

Digital Library

[29]

Chris Harrison, Shilpa Ramamurthy, and Scott E. Hudson. 2012. On-Body Interaction: Armed and Dangerous. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction(TEI ’12). Association for Computing Machinery, New York, NY, USA, 69–76. https://doi.org/10.1145/2148131.2148148

Digital Library

[30]

Tom Horak, Sriram Karthik Badam, Niklas Elmqvist, and Raimund Dachselt. 2018. When David Meets Goliath: Combining Smartwatches with a Large Vertical Display for Visual Data Exploration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, New York, New York, USA, 1–13. https://doi.org/10.1145/3173574.3173593

Digital Library

[31]

Christophe Hurter, Nathalie Henry Riche, Steven M. Drucker, Maxime Cordeil, Richard Alligier, and Romain Vuillemot. 2019. FiberClay: Sculpting Three Dimensional Trajectories to Reveal Structural Insights. IEEE Transactions on Visualization and Computer Graphics 25, 1 (Jan. 2019), 704–714. https://doi.org/10.1109/TVCG.2018.2865191

Digital Library

[32]

Robert J. K. Jacob. 1990. What You Look at Is What You Get: Eye Movement-Based Interaction Techniques. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People - CHI ’90, Vol. 23. ACM Press, New York, New York, USA, 11–18. https://doi.org/10.1145/97243.97246

Digital Library

[33]

Daniel A. Keim. Jan.-March/2002. Information Visualization and Visual Data Mining. IEEE Transactions on Visualization and Computer Graphics 8, 1(Jan.-March/2002), 1–8. https://doi.org/10.1109/2945.981847

Digital Library

[34]

Matthias Kraus, Katrin Angerbauer, Juri Buchmüller, Daniel Schweitzer, Daniel A. Keim, Michael Sedlmair, and Johannes Fuchs. 2020. Assessing 2D and 3D Heatmaps for Comparative Analysis: An Empirical Study. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, Honolulu, HI, USA, 1–14. https://doi.org/10.1145/3313831.3376675

Digital Library

[35]

M. Kraus, N. Weiler, D. Oelke, J. Kehrer, D. A. Keim, and J. Fuchs. 2019. The Impact of Immersion on Cluster Identification Tasks. IEEE Transactions on Visualization and Computer Graphics 26, 1 (jan 2019), 1–1. https://doi.org/10.1109/TVCG.2019.2934395

[36]

Ricardo Langner, Tom Horak, and Raimund Dachselt. 2018. VisTiles: Coordinating and Combining Co-Located Mobile Devices for Visual Data Exploration. IEEE Transactions on Visualization and Computer Graphics 24, 1 (jan 2018), 626–636. https://doi.org/10.1109/TVCG.2017.2744019

[37]

Joseph J. LaViola, Ernst Kruijff, Ryan P. McMahan, Doug A. Bowman, and Ivan Poupyrev. 2017. 3D User Interfaces: Theory and Practice (second edition ed.). Addison-Wesley, Boston.

[38]

Huy Viet Le, Sven Mayer, and Niels Henze. 2018. InfiniTouch: Finger-Aware Interaction on Fully Touch Sensitive Smartphones. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology(UIST ’18). Association for Computing Machinery, New York, NY, USA, 779–792. https://doi.org/10.1145/3242587.3242605

Digital Library

[39]

Bongshin Lee, Eun Kyoung Choe, Petra Isenberg, Kim Marriott, and John Stasko. 2020. Reaching Broader Audiences With Data Visualization. IEEE Computer Graphics and Applications 40, 2 (March 2020), 82–90. https://doi.org/10.1109/MCG.2020.2968244

Digital Library

[40]

Benjamin Lee, Xiaoyun Hu, Maxime Cordeil, Arnaud Prouzeau, Bernhard Jenny, and Tim Dwyer. 2020. Shared Surfaces and Spaces: Collaborative Data Visualisation in a Co-Located Immersive Environment. IEEE Transactions on Visualization and Computer Graphics (Aug. 2020), 1–1. https://doi.org/10.1109/tvcg.2020.3030450

[41]

Bongshin Lee, Arjun Srinivasan, John Stasko, Melanie Tory, and Vidya Setlur. 2018. Multimodal Interaction for Data Visualization. In Proceedings of the 2018 International Conference on Advanced Visual Interfaces - AVI ’18(Proceedings of the 2018 International Conference on Advanced Visual Interfaces - AVI ’18). ACM, 1–3. https://doi.org/10.1145/3206505.3206602

Digital Library

[42]

Chi-Jung Lee and Hung-Kuo Chu. 2018. Dual-MR: Interaction with Mixed Reality Using Smartphones. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology(VRST ’18). Association for Computing Machinery, Tokyo, Japan, 1–2. https://doi.org/10.1145/3281505.3281618

Digital Library

[43]

Yiqin Lu, Chun Yu, Xin Yi, Yuanchun Shi, and Shengdong Zhao. 2017. BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb’s Muscle Memory. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (June 2017), 1–24. https://doi.org/10.1145/3090083

Digital Library

[44]

Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H. Thomas (Eds.). 2018. Immersive Analytics. Number 11190 in Lecture Notes in Computer Science. Springer, Cham.

[45]

Tamara Munzner. 2014. Visualization Analysis and Design. A K Peters/CRC Press, Boca Raton, FL. https://doi.org/10.1201/b17511

[46]

Ian Oakley and Junseok Park. 2009. Motion Marking Menus: An Eyes-Free Approach to Motion Input for Handheld Devices. International Journal of Human-Computer Studies 67, 6 (June 2009), 515–532. https://doi.org/10.1016/j.ijhcs.2009.02.002

Digital Library

[47]

Sharon Oviatt. 1999. Ten Myths of Multimodal Interaction. Commun. ACM 42, 11 (nov 1999), 74–81. https://doi.org/10.1145/319382.319398

Digital Library

[48]

Sharon Oviatt and Philip Cohen. 2000. Perceptual User Interfaces: Multimodal Interfaces That Process What Comes Naturally. Commun. ACM 43, 3 (March 2000), 45–53. https://doi.org/10.1145/330534.330538

Digital Library

[49]

Sharon Oviatt, Rebecca Lunsford, and Rachel Coulston. 2005. Individual Differences in Multimodal Integration Patterns: What Are They and Why Do They Exist?. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’05(Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’05). ACM Press, 241–249. Issue April. https://doi.org/10.1145/1054972.1055006

Digital Library

[50]

Simon T. Perrault, Eric Lecolinet, James Eagan, and Yves Guiard. 2013. Watchit: Simple Gestures and Eyes-Free Interaction for Wristwatches and Bracelets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’13). Association for Computing Machinery, New York, NY, USA, 1451–1460. https://doi.org/10.1145/2470654.2466192

Digital Library

[51]

Arnaud Prouzeau, Antoine Lhuillier, Barrett Ens, Daniel Weiskopf, and Tim Dwyer. 2019. Visual Link Routing in Immersive Visualisations. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces(ISS ’19). Association for Computing Machinery, New York, NY, USA, 241–253. https://doi.org/10.1145/3343055.3359709

Digital Library

[52]

Patrick Reipschläger and Raimund Dachselt. 2019. DesignAR: Immersive 3D-Modeling Combining Augmented Reality with Interactive Displays. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces - ISS ’19. ACM Press, Daejeon, Republic of Korea, 29–41. https://doi.org/10.1145/3343055.3359718

Digital Library

[53]

Patrick Reipschläger, Tamara Flemisch, and Raimund Dachselt. 2020. Personal Augmented Reality for Information Visualization on Large Interactive Displays. IEEE Transactions on Visualization and Computer Graphics (Feb. 2020), 1–1. https://doi.org/10.1109/tvcg.2020.3030460

[54]

Ayshwarya Saktheeswaran, Arjun Srinivasan, and John Stasko. 2020. Touch? Speech? Or Touch and Speech? Investigating Multimodal Interaction for Visual Network Exploration and Analysis. IEEE Transactions on Visualization and Computer Graphics 26, 6 (jun 2020), 1–1. https://doi.org/10.1109/TVCG.2020.2970512

[55]

Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Construction of a Benchmark for the User Experience Questionnaire (UEQ). International Journal of Interactive Multimedia and Artificial Intelligence 4, 4 (2017), 40. https://doi.org/10.9781/ijimai.2017.445

[56]

Ronell Sicat, Jiabao Li, Junyoung Choi, Maxime Cordeil, Won-Ki Jeong, Benjamin Bach, and Hanspeter Pfister. 2019. DXR: A Toolkit for Building Immersive Data Visualizations. IEEE Transactions on Visualization and Computer Graphics 25, 1 (Jan. 2019), 715–725. https://doi.org/10.1109/TVCG.2018.2865152

Digital Library

[57]

Maurício Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simões Lopes. 2017. VRRRRoom: Virtual Reality for Radiologists in the Reading Room. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’17). ACM Press, New York, NY, USA, 4057–4062. https://doi.org/10.1145/3025453.3025566

Digital Library

[58]

Arjun Srinivasan, Bongshin Lee, Nathalie Henry Riche, Steven M. Drucker, and Ken Hinckley. 2020. InChorus: Designing Consistent Multimodal Interactions for Data Visualization on Tablet Devices. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, Honolulu, HI, USA, 1–13. https://doi.org/10.1145/3313831.3376782

Digital Library

[59]

Arjun Srinivasan and John Stasko. 2018. Orko: Facilitating Multimodal Interaction for Visual Exploration and Analysis of Networks. IEEE Transactions on Visualization and Computer Graphics 24, 1 (jan 2018), 1–1. https://doi.org/10.1109/TVCG.2017.2745219

[60]

Seung Youb Ssin, James A. Walsh, Ross T. Smith, Andrew Cunningham, and Bruce H. Thomas. 2019. GeoGate: Correlating Geo-Temporal Datasets Using an Augmented Reality Space-Time Cube and Tangible Interactions. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Osaka, Japan, 210–219. https://doi.org/10.1109/VR.2019.8797812

[61]

Hemant Bhaskar Surale, Aakar Gupta, Mark Hancock, and Daniel Vogel. 2019. TabletInVR: Exploring the Design Space for Using a Multi-Touch Tablet in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19. ACM Press, Glasgow, Scotland Uk, 1–13. https://doi.org/10.1145/3290605.3300243

Digital Library

[62]

Zsolt Szalavári and Michael Gervautz. 2008. The Personal Interaction Panel - a Two-Handed Interface for Augmented Reality. Computer Graphics Forum 16 (June 2008), C335–C346. https://doi.org/10.1111/1467-8659.16.3conferenceissue.35

[63]

Slawomir Konrad Tadeja, Timoleon Kipouros, and Per Ola Kristensson. 2019. Exploring Parallel Coordinates Plots in Virtual Reality. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI EA ’19. ACM Press, Glasgow, Scotland Uk, 1–6. https://doi.org/10.1145/3290607.3313068

Digital Library

[64]

Philipp Wacker, Oliver Nowak, Simon Voelker, and Jan Borchers. 2019. ARPen: Mid-Air Object Manipulation Techniques for a Bimanual AR System with Pen & Smartphone. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, Glasgow, Scotland Uk, 1–12. https://doi.org/10.1145/3290605.3300849

Digital Library

[65]

Matt Whitlock, Stephen Smart, and Danielle Albers Szafir. 2020. Graphical Perception for Immersive Analytics. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Atlanta, GA, USA, 616–625. https://doi.org/10.1109/VR46266.2020.00084

[66]

Pui Chung Wong, Kening Zhu, Xing-Dong Yang, and Hongbo Fu. 2020. Exploring Eyes-Free Bezel-Initiated Swipe on Round Smartwatches. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, Honolulu, HI, USA, 1–11. https://doi.org/10.1145/3313831.3376393

Digital Library

[67]

Yalong Yang, Tim Dwyer, Bernhard Jenny, Kim Marriott, Maxime Cordeil, and Haohui Chen. 2018. Origin-Destination Flow Maps in Immersive Environments. IEEE Transactions on Visualization and Computer Graphics 25, 1 (jan 2018), 1–1. https://doi.org/10.1109/TVCG.2018.2865192

Digital Library

[68]

Bo Yi, Xiang Cao, Morten Fjeld, and Shengdong Zhao. 2012. Exploring User Motivations for Eyes-Free Interaction on Mobile Devices. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems - CHI ’12. ACM Press, Austin, Texas, USA, 2789. https://doi.org/10.1145/2207676.2208678

Digital Library

[69]

Fengyuan Zhu and Tovi Grossman. 2020. BISHARE: Exploring Bidirectional Interactions Between Smartphones and Head-Mounted Augmented Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, Honolulu, HI, USA, 1–14. https://doi.org/10.1145/3313831.3376233

Digital Library

Cited By

Zhu QWang ZZeng WTong WLin WMa X(2024)Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public EnvironmentsProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642049(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642049
Friedl‐Knirsch JPointecker FPfistermüller SStach CAnthes CRoth D(2024)A Systematic Literature Review of User Evaluation in Immersive AnalyticsComputer Graphics Forum10.1111/cgf.1511143:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15111
Quijano-Chavez CDoerr NLee BSchmalstieg DSedlmair M(2024)Brushing and Linking for Situated Analytics2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00116(597-603)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00116
Show More Cited By

Index Terms

STREAM: Exploring the Combination of Spatially-Aware Tablets with Augmented Reality Head-Mounted Displays for Immersive Analytics
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Index terms have been assigned to the content through auto-classification.

Recommendations

Tangible Globes for Data Visualisation in Augmented Reality
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Head-mounted augmented reality (AR) displays allow for the seamless integration of virtual visualisation with contextual tangible references, such as physical (tangible) globes. We explore the design of immersive geospatial data visualisation with AR and ...
Read More
Augmented Displays: Seamlessly Extending Interactive Surfaces With Head-Mounted Augmented Reality
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

We present Augmented Displays, a new class of display systems combining high-resolution interactive surfaces with head-coupled Augmented Reality. This extends the screen estate beyond the display and enables placing AR content directly at the display's ...
Read More
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
2,228
Total Downloads

Downloads (Last 12 months)621
Downloads (Last 6 weeks)55

Other Metrics

View Author Metrics

Citations

Cited By

Zhu QWang ZZeng WTong WLin WMa X(2024)Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public EnvironmentsProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642049(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642049
Friedl‐Knirsch JPointecker FPfistermüller SStach CAnthes CRoth D(2024)A Systematic Literature Review of User Evaluation in Immersive AnalyticsComputer Graphics Forum10.1111/cgf.1511143:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15111
Quijano-Chavez CDoerr NLee BSchmalstieg DSedlmair M(2024)Brushing and Linking for Situated Analytics2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00116(597-603)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00116
Friedl-Knirsch JStach CPointecker FAnthes CRoth D(2024)A Study on Collaborative Visual Data Analysis in Augmented Reality with Asymmetric Display TypesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337210330:5(2633-2643)Online publication date: May-2024
https://doi.org/10.1109/TVCG.2024.3372103
Grubert JWitzani LOtte AGesslein TKranz MKristensson P(2024)Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone SystemIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.330931630:8(5830-5846)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3309316
Lunding MGrønbæk JGrymer NWells THouben SPetersen M(2023)Reality and Beyond: Proxemics as a Lens for Designing Handheld Collaborative Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36264637:ISS(21-40)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626463
Iquiapaza YWagner JNedel L(2023)DeAR: Combining Desktop and Augmented Reality for Visual Data AnalysisProceedings of the 25th Symposium on Virtual and Augmented Reality10.1145/3625008.3625021(233-237)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625008.3625021
Auda JGruenefeld UFaltaous SMayer SSchneegass S(2023)A Scoping Survey on Cross-reality SystemsACM Computing Surveys10.1145/361653656:4(1-38)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3616536
Ghaemi ZEns BEngelke UJenny B(2023)Drawing Connections: Designing Situated Links for Immersive MapsProceedings of the ACM on Human-Computer Interaction10.1145/36042527:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604252
Chulpongsatorn NWillett WSuzuki R(2023)HoloTouch: Interacting with Mixed Reality Visualizations Through Smartphone ProxiesExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585738(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585738
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents