research-article

Interaction with Virtual Content using Augmented Reality: A User Study in Assembly Procedures

Authors:

Bernardo Marques,

Carlos Ferreira,

Beatriz Sousa SantosAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 4, Issue ISS

Article No.: 196, Pages 1 - 17

https://doi.org/10.1145/3427324

Published: 04 November 2020 Publication History

Abstract

Assembly procedures are a common task in several domains of application. Augmented Reality (AR) has been considered as having great potential in assisting users while performing such tasks. However, poor interaction design and lack of studies, often results in complex and hard to use AR systems. This paper considers three different interaction methods for assembly procedures (Touch gestures in a mobile device; Mobile Device movements; 3D Controllers and See-through HMD). It also describes a controlled experiment aimed at comparing acceptance and usability between these methods in an assembly task using Lego blocks. The main conclusions are that participants were faster using the 3D controllers and Video see-through HMD. Participants also preferred the HMD condition, even though some reported light symptoms of nausea, sickness and/or disorientation, probably due to limited resolution of the HMD cameras used in the video see-through setting and some latency issues. In addition, although some research claims that manipulation of virtual objects with movements of the mobile device can be considered as natural, this condition was the least preferred by the participants.

Supplementary Material

MP4 File (v4issa196.mp4)

Supplemental video

Download
38.52 MB

References

[1]

Jo a o Alves, Bernardo Marques, Miguel Neves, Rafael Maio, André Santos, Inê s Justo, Raquel Rainho, Dany Costa, Beatriz Sousa Santos, and Paulo Dias. 2019. Augmented Reality Platform to foster Collaborative Manipulation of 3D virtual objects: A Prototype. In International Conference on Graphics and Interaction, ICGI 2019.

[2]

J. Alves, B. Marques, M. Oliveira, T. Araújo, P. Dias, and B. S. Santos. 2019. Comparing Spatial and Mobile Augmented Reality for Guiding Assembling Procedures with Task Validation. In 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC). 1--6.

[3]

Ronald T. Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, Vol. 6, 4 (1997).

[4]

R. T. Azuma. 2016. The Most Important Challenge Facing Augmented Reality. Presence, Vol. 25, 3 (Dec 2016), 234--238.

Digital Library

[5]

Mafkereseb Kassahun Bekele, Roberto Pierdicca, Emanuele Frontoni, Eva Savina Malinverni, and James Gain. 2018. A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage. J. Comput. Cult. Herit., Vol. 11, 2 (March 2018), 7:1--7:36.

Digital Library

[6]

Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality . Foundations and Trends in Human--Computer Interaction, Vol. 8, 2--3 (2015), 73--272.

Digital Library

[7]

Mark Billinghurst and Hirokazu Kato. 2002. Collaborative Augmented reality. In Communications of the ACM .

[8]

Mark Billinghurst, Hiroyuki Kato, and Ivan Poupyrev. 2008. Tangible augmented reality. In SIGGRAPH 2008.

[9]

Volkert Buchmann, Stephen Violich, Mark Billinghurst, and Andy Cockburn. 2004. FingARtips: Gesture Based Direct Manipulation in Augmented Reality. In Proceedings of the 2Nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (Singapore) (GRAPHITE '04). ACM, New York, NY, USA, 212--221.

Digital Library

[10]

Sebastian Büttner, Markus Funk, Oliver Sand, and Carsten Röcker. 2016. Using Head-Mounted Displays and In-Situ Projection for Assistive Systems: A Comparison. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Island, Greece) (PETRA '16). ACM, New York, NY, USA, Article 44, bibinfonumpages8 pages.

Digital Library

[11]

Fabio M. Caputo, Marco Emporio, and Andrea Giachetti. 2018. The Smart Pin: An effective tool for object manipulation in immersive virtual reality environments. Computers & Graphics, Vol. 74 (2018), 225 -- 233.

[12]

D. Chatzopoulos, C. Bermejo, Z. Huang, and P. Hui. 2017. Mobile Augmented Reality Survey: From Where We Are to Where We Go. IEEE Access, Vol. 5 (2017), 6917--6950.

[13]

Arindam Dey, Mark Billinghurst, Robert W Lindeman, and J Edward Swan Ii. 2018. A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 2014 . Frontiers in Robotics and AI, Vol. 5 (2018).

[14]

Paul Dizio and James Lackner. 2000. Motion Sickness Side Effects and Aftereffects of Immersive Virtual Environments Created with Helmet-Mounted Visual Displays. (11 2000), 4.

[15]

Andreas Dü nser and Mark Billinghurst. 2011. Evaluating Augmented Reality Systems . In Handbook of Augmented Reality . 289--307.

[16]

I n igo Ferná ndez Del Amo, John Ahmet Erkoyuncu, Rajkumar Roy, Riccardo Palmarini, and Demetrius Onoufriou. 2018. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications . Computers in Industry, Vol. 103 (2018), 47--71.

[17]

Lawrence H. Frank, John G. Casali, and Walter W. Wierwille. 1988. Effects of Visual Display and Motion System Delays on Operator Performance and Uneasiness in a Driving Simulator. Human Factors, Vol. 30, 2 (1988), 201--217.

Digital Library

[18]

Markus Funk, Andreas B"achler, Liane B"achler, Thomas Kosch, Thomas Heidenreich, and Albrecht Schmidt. 2017. Working with Augmented Reality? A Long-Term Analysis of In-Situ Instructions at the Assembly Workplace. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '17). Association for Computing Machinery, New York, NY, USA, 222--229.

Digital Library

[19]

Jean Dickinson Gibbons and Subhabrata Chakraborti. 2011. Nonparametric Statistical Inference .Springer Berlin Heidelberg, Berlin, Heidelberg, 977--979.

[20]

E. S. Goh, M. S. Sunar, and A. W. Ismail. 2019. 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review. IEEE Access, Vol. 7 (2019), 40581--40601.

[21]

Jens Grubert and Stefanie Zollmann. 2017. Towards Pervasive Augmented Reality: Context- Awareness in Augmented Reality . IEEE Transactions on Visualization and Computer Graphics, Vol. 23, 6 (2017), 1706--1724.

Digital Library

[22]

Devamardeep Hayatpur, Seongkook Heo, Haijun Xia, Wolfgang Stuerzlinger, and Daniel Wigdor. 2019. Plane, Ray, and Point: Enabling Precise Spatial Manipulations with Shape Constraints. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST '19). Association for Computing Machinery, New York, NY, USA, 1185--1195.

Digital Library

[23]

Anders Henrysson, Mark Billinghurst, and Mark Ollila. 2005. Face to face collaborative AR on mobile phones . IEEE a International Symposium on Symposium on Mixed and Augmented Reality, ISMAR 2005 (2005), 80--89.

[24]

David C. Hoaglin, Frederick Mosteller, and John W. Tukey (Editor). 2000. Understanding Robust and Exploratory Data Analysis 1 ed.). Wiley-Interscience.

[25]

Jé rme Jetter, Jö rgen Eimecke, and Alexandra Rese. 2018. Augmented reality tools for industrial applications: What are potential key performance indicators and who benefits? Computers in Human Behavior, Vol. 87 (2018), 18--33.

Digital Library

[26]

B. M. Khuong, K. Kiyokawa, A. Miller, J. J. La Viola, T. Mashita, and H. Takemura. 2014. The effectiveness of an AR-based context-aware assembly support system in object assembly. In 2014 IEEE Virtual Reality (VR) . 57--62.

[27]

K. Kim, M. Billinghurst, G. Bruder, H. B. Duh, and G. F. Welch. 2018. Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008--2017). IEEE Transactions on Visualization and Computer Graphics, Vol. 24, 11 (2018), 2947--2962.

[28]

D W F Van Krevelen and R Poelman. 2010. A Survey of Augmented Reality Technologies, Applications and Limitations . The International Journal of Virtual Reality, Vol. 9 (2010).

[29]

M. Krichenbauer, G. Yamamoto, T. Taketom, C. Sandor, and H. Kato. 2018. Augmented Reality versus Virtual Reality for 3D Object Manipulation. IEEE Transactions on Visualization and Computer Graphics, Vol. 24, 2 (Feb 2018), 1038--1048.

[30]

Ze-Hao Lai, Wenjin Tao, Ming C. Leu, and Zhaozheng Yin. 2020. Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. Journal of Manufacturing Systems, Vol. 55 (2020), 69 -- 81.

[31]

Joseph J. LaViola Jr., Ernst Kruijff, Doug Bowman, Ivan P. Poupyrev, and Ryan P. McMahan. 2017. 3D User Interfaces: Theory and Practice (second edition) .

[32]

Wenkai Li, A. Nee, and S. Ong. 2017. A State-of-the-Art Review of Augmented Reality in Engineering Analysis and Simulation . Multimodal Technologies and Interaction, Vol. 1, 3 (2017).

[33]

Jingbo Liu, Oscar Kin-Chung Au, Hongbo Fu, and Chiew-Lan Tai. 2012. Two-Finger Gestures for 6DOF Manipulation of 3D Objects. Comput. Graph. Forum, Vol. 31, 7pt1 (Sept. 2012), 2047--2055.

Digital Library

[34]

F. Loch, F. Quint, and I. Brishtel. 2016. Comparing Video and Augmented Reality Assistance in Manual Assembly. In 2016 12th International Conference on Intelligent Environments (IE). 147--150.

[35]

Annette Mossel, Benjamin Venditti, and Hannes Kaufmann. 2013. 3DTouch and HOMER-S: Intuitive Manipulation Techniques for One-Handed Handheld Augmented Reality. In Proceedings of the Virtual Reality International Conference: Laval Virtual. Association for Computing Machinery, New York, NY, USA, 10.

Digital Library

[36]

K. Murakami, R. Kiyama, T. Narumi, T. Tanikawa, and M. Hirose. 2013. Poster: A wearable augmented reality system with haptic feedback and its performance in virtual assembly tasks. In 2013 IEEE Symposium on 3D User Interfaces (3DUI) . 161--162.

[37]

A. Nishihara and J. Okamoto. 2015. Object recognition in assembly assisted by augmented reality system. In 2015 SAI Intelligent Systems Conference (IntelliSys). 400--407.

[38]

Michael E. Porter and James E. Heppelmann. 2017. A Manager's Guide to Augmented Reality. Harvard Business Review. Technical Report.

[39]

Moritz Quandt, Benjamin Knoke, Christian Gorldt, Michael Freitag, and Klaus Dieter Thoben. 2018. General Requirements for Industrial Augmented Reality Applications . Procedia CIRP, Vol. 72 (2018), 1130--1135.

[40]

Rafael Radkowski and Christian Stritzke. 2012. Interactive Hand Gesture-based Assembly for Augmented Reality Applications., bibinfonumpages5 pages.

[41]

Christian Sandor, Martin Fuchs, Alvaro Cassinelli, Hao Li, Richard Newcombe, Goshiro Yamamoto, and Steven Feiner. 2015. Breaking the Barriers to True Augmented Reality . ArXiv e-prints, Vol. 1512.05471 (2015), 1--13.

[42]

C.Y. Siew, S.K. Ong, and A.Y.C. Nee. 2019. A practical augmented reality-assisted maintenance system framework for adaptive user support. Robotics and Computer-Integrated Manufacturing, Vol. 59 (2019), 115 -- 129.

Digital Library

[43]

B.G. Tabachnick and L.S. Fidell. 2007. Experimental Designs Using ANOVA .Thomson/Brooks/Cole.

[44]

Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative Effectiveness of Augmented Reality in Object Assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). ACM, New York, NY, USA, 73--80.

Digital Library

[45]

Matthew Turk. 2014. Multimodal interaction: A review. Pattern Recognition Letters, Vol. 36 (2014), 189 -- 195.

Digital Library

[46]

Peng Wang, Shusheng Zhang, Mark Billinghurst, Xiaoliang Bai, Weiping He, Shuxia Wang, Mengmeng Sun, and Xu Zhang. 2019. A comprehensive survey of AR/MR-based co-design in manufacturing . Engineering with Computers (2019), 1--24.

[47]

Xiangyu Wang, Soh K Ong, and Andrew YC Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing, Vol. 4, 1 (2016), 1--22.

[48]

Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems, Vol. 61, 4 (2013), 398 -- 403. Models and Technologies for Multi-modal Skill Training.

Digital Library

[49]

M. L. Yuan, S. K. Ong, and A. Y. C. Nee. 2008. Augmented reality for assembly guidance using a virtual interactive tool. International Journal of Production Research, Vol. 46, 7 (2008), 1745--1767.

[50]

Feng Zhou, Henry Duh, and Mark Billinghurst. 2008. Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR . IEEE International Symposium on Symposium on Mixed and Augmented Reality, ISMAR 2008 (2008), 193--202.

Digital Library

Cited By

Zhang VAlbers ASaeedi-Givi CKristensson PBohné TTadeja S(2024)Should I Evaluate My Augmented Reality System in an Industrial Environment? Investigating the Effects of Classroom and Shop Floor Settings on Guided AssemblyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345620830:11(7042-7052)Online publication date: Nov-2024
https://doi.org/10.1109/TVCG.2024.3456208
Wang BZheng LWang YFang WWang L(2024)Towards the industry 5.0 frontier: Review and prospect of XR in product assemblyJournal of Manufacturing Systems10.1016/j.jmsy.2024.05.00274(777-811)Online publication date: Jun-2024
https://doi.org/10.1016/j.jmsy.2024.05.002
Peixe LAgati SHounsell M(2023)Manual Assembly Augmented Reality Systems Implementation: A Systematic Literature MappingProceedings of the 25th Symposium on Virtual and Augmented Reality10.1145/3625008.3625011(17-25)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625008.3625011
Show More Cited By

Index Terms

Interaction with Virtual Content using Augmented Reality: A User Study in Assembly Procedures
1. Applied computing
  1. Operations research
    1. Computer-aided manufacturing
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Usability testing
      2. User studies
    2. Interaction paradigms
      1. Mixed / augmented reality

Recommendations

Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Multi-sensorial virtual reality and augmented human food interaction
MHFI '16: Proceedings of the 1st Workshop on Multi-sensorial Approaches to Human-Food Interaction

In the field of virtual reality (VR) research, media technologies to create a realistic feeling of being present in a real/virtual world by duplicating multi-sensory information have been studied over a long period. Recently, technologies for multi-...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 4, Issue ISS

ISS

November 2020

488 pages

EISSN:2573-0142

DOI:10.1145/3433930

Editor:
Jeff Nichols
Apple Inc.

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 November 2020

Published in PACMHCI Volume 4, Issue ISS

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

IEETA - Institute of Electronics and Informatics Engineering of Aveiro, funded by FCT, project [UID/CEC/00127/2019]
Produtech ? SIF ? Solutions for the Industry of the Future [POCI-01- 0247-FEDER-024541]

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
790
Total Downloads

Downloads (Last 12 months)119
Downloads (Last 6 weeks)9

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang VAlbers ASaeedi-Givi CKristensson PBohné TTadeja S(2024)Should I Evaluate My Augmented Reality System in an Industrial Environment? Investigating the Effects of Classroom and Shop Floor Settings on Guided AssemblyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345620830:11(7042-7052)Online publication date: Nov-2024
https://doi.org/10.1109/TVCG.2024.3456208
Wang BZheng LWang YFang WWang L(2024)Towards the industry 5.0 frontier: Review and prospect of XR in product assemblyJournal of Manufacturing Systems10.1016/j.jmsy.2024.05.00274(777-811)Online publication date: Jun-2024
https://doi.org/10.1016/j.jmsy.2024.05.002
Peixe LAgati SHounsell M(2023)Manual Assembly Augmented Reality Systems Implementation: A Systematic Literature MappingProceedings of the 25th Symposium on Virtual and Augmented Reality10.1145/3625008.3625011(17-25)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625008.3625011
Karg PStöhr RJonas LKreimeier JGötzelmann T(2023)Reflect-AR: Insights into Mirror-Based Augmented Reality Instructions to Support Manual Assembly TasksProceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3594806.3594866(62-68)Online publication date: 5-Jul-2023
https://dl.acm.org/doi/10.1145/3594806.3594866
Spittle BFrutos-Pascual MCreed CWilliams I(2023)A Review of Interaction Techniques for Immersive EnvironmentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317480529:9(3900-3921)Online publication date: 1-Sep-2023
https://doi.org/10.1109/TVCG.2022.3174805
Maio RMarques BAlves JSantos BDias PLau N(2022)An Augmented Reality Serious Game for Learning Intelligent Wheelchair Control: Comparing Configuration and Tracking MethodsSensors10.3390/s2220778822:20(7788)Online publication date: 13-Oct-2022
https://doi.org/10.3390/s22207788
Madeira TMarques BNeves PDias PSantos B(2022)Comparing Desktop vs. Mobile Interaction for the Creation of Pervasive Augmented Reality ExperiencesJournal of Imaging10.3390/jimaging80300798:3(79)Online publication date: 18-Mar-2022
https://doi.org/10.3390/jimaging8030079
Azofeifa JNoguez JRuiz SMolina-Espinosa JMagana ABenes B(2022)Systematic Review of Multimodal Human–Computer InteractionInformatics10.3390/informatics90100139:1(13)Online publication date: 15-Feb-2022
https://doi.org/10.3390/informatics9010013
Gattullo MLaviola EEvangelista AFiorentino MUva A(2022)Towards the Evaluation of Augmented Reality in the Metaverse: Information Presentation ModesApplied Sciences10.3390/app12241260012:24(12600)Online publication date: 8-Dec-2022
https://doi.org/10.3390/app122412600
Maio RAlves JMarques BDias PLau NBottoni PPanizzi E(2022)Exploring an Augmented Reality Serious Game for Motorized Wheelchair ControlProceedings of the 2022 International Conference on Advanced Visual Interfaces10.1145/3531073.3534471(1-3)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1145/3531073.3534471
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents