research-article

Use of wearable and augmented reality technologies in industrial maintenance work

Authors:

Susanna Aromaa,

Ilari ParkkinenAuthors Info & Claims

AcademicMindtrek '16: Proceedings of the 20th International Academic Mindtrek Conference

Pages 235 - 242

https://doi.org/10.1145/2994310.2994321

Published: 17 October 2016 Publication History

Abstract

Industrial maintenance is an increasingly complex and knowledge intensive field. Although new technologies in maintenance have been studied extensively, their usage is still lacking in the industry. We have studied knowledge-sharing solutions using augmented reality (AR) and wearable technologies in actual industry cases to find out if maintenance technicians find them useful and usable in their everyday work. Two test cases were included: the use of a wearable system consisting of three devices in the crane industry, and the use of AR guidance in the marine industry. In both cases two maintenance technicians tested the technologies and data were collected using questionnaires, interviews and observation. The maintenance technicians were positive towards the use of these technologies in their work. However, some practical issues were raised concerning the simultaneous use of multiple devices and the placement of the devices. A more system-level approach to designing wearable and AR technologies could be applied to ensure their utility in the field. Findings from this study can be used when designing and implementing wearable and AR technologies in maintenance, but also in other industry domains like the manufacturing industry.

References

[1]

Aromaa, S., Väätänen, A., Aaltonen, I., and Heimonen, T. 2015. A model for gathering and sharing knowledge in maintenance work. In Proceedings of the 33rd Annual Conference of the European Association of Cognitive Ergonomics (Warsaw, Poland, July 1--3, 2015). ECCE '15, Article 28.

Digital Library

[2]

European Committee for Standardization 2010. EN 13306.

[3]

Reason, J. 1997. Managing the Risks of Organizational Accidents. Ashgate Publishing Ltd.

[4]

Franssila, H. 2008. Franssila, Sähköisen tietämyksenhallinnan mukauttaminen ja integrointi työkäytäntöihin asennus- ja huoltotyössä (Integrating digital knowledge management in assembly and maintenance work).

[5]

Starner, T., Mann, S., Rhodes, B., Levine, J., Healey, J., Kirsch, D., Picard, R., and Pentland, A. 1997. Augmented Reality through Wearable Computing. Presence Teleoperators Virtual Environ. 6, 4, 386--398.

Digital Library

[6]

Mann, S. 1997. Wearable computing: A first step toward personal imaging. Computer (Long. Beach. Calif). 30, 2, 25--32. DOI=http://doi.org/10.1109/2.566147.

Digital Library

[7]

Billinghurst, M. 1999. New ways to manage information. Computer (Long. Beach. Calif). 32, 1, 57--64. DOI=http://doi.org/10.1109/2.738305.

Digital Library

[8]

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. 2001. Recent Advances in. Comput. Graph. Appl. 21, 6, 34--47. DOI=http://doi.org/10.1007/978-81-322-2235-4.

Digital Library

[9]

Barfield, W. (Ed.). 2015. Fundamentals of wearable computers and augmented reality. CRC Press.

[10]

Lukowicz, P., Timm-Giel, A., Lawo, M., and Herzog, O. 2007. WearIT@work: Toward Real-World Industrial Wearable Computing. Pervasive Computing. 6, 4, 8--13.

Digital Library

[11]

Laarni, J., Heinilä, J., Häkkinen, J., Kalakoski, V., Kallinen, K., Lukander, K., Löppönen, P., Palomäki, T., Ravaja, N., Savioja, P., and Väätänen, A. 2009. Supporting situation awareness in demanding operating environments through wearable user interfaces. In 8th International Conference of Engineering Psychology and Cognitive Ergonomics (EPCE) (San Diego, CA, USA, July 19 -- 24, 2009), 13--21. DOI=http://doi.org/10.1007/978-3-642-02728-4_2.

Digital Library

[12]

Elliott, L. R., Van Erp, J., Redden, E. S., and Duistermaat, M. 2010. Field-based validation of a tactile navigation device. IEEE Trans. Haptics. 3, 2, 78--87. DOI=http://doi.org/10.1109/TOH.2010.3.

Digital Library

[13]

Wang, X., Ong, S. K., and Nee, A. Y. C. 2016. A comprehensive survey of augmented reality assembly research. Adv. Manuf. 4, 1, 1--22. DOI=http://doi.org/10.1007/s40436-015-0131-4.

[14]

Nee, A. Y. C., Ong, S. K., Chryssolouris, G., and Mourtzis, D. 2012. Augmented reality applications in design and manufacturing. CIRP Ann. - Manuf. Technol. 61, 2, 657--679. DOI=http://doi.org/10.1016/j.cirp.2012.05.010.

[15]

Ong, S. K., Yuan, M. L., and Nee, A. Y. C. 2008. Augmented reality applications in manufacturing: a survey. Int. J. Prod. Res. 46, 10, 2707--2742. DOI=http://doi.org/10.1080/00207540601064773.

[16]

Aromaa, S., Aaltonen, I., and Väätänen, A. 2016. Technology Concepts to Improve Knowledge Sharing During Maintenance. In The Ninth International Conference on Advances in Computer-Human Interactions (Venice, Italy April 24--28, 2016). ACHI'16.

[17]

Baird, K. M., and Barfield, W. 1999. Evaluating the Effectiveness of Augmented Reality Displays for a Manual Assembly Task. Virtual Real. 4, 250--259.

Digital Library

[18]

Day, P. N., Ferguson, G., Holt, P. O. B., Hogg, S., and Gibson, D. 2005. Wearable augmented virtual reality for enhancing information delivery in high precision defence assembly: An engineering case study. Virtual Real. 8, 3, 177--184. DOI=http://doi.org/10.1007/s10055-004-0147-8.

Digital Library

[19]

Kunze, K., Wagner, F., and Kartal, E. 2009. Does context matter?-a quantitative evaluation in a real world maintenance scenario. Pervasive Comput. 372--389.

Digital Library

[20]

Henderson, S. J., and Feiner, S. 2009. Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In Science and Technology Proceedings - IEEE 2009 International Symposium on Mixed and Augmented Reality, ISMAR 2009 (Orlando, Florida, USA, October 19 -- 22, 2009), 135--144. DOI=http://doi.org/10.1109/ISMAR.2009.5336486.

Digital Library

[21]

Henderson, S., and Feiner, S. 2011. Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans. Vis. Comput. Graph. 17, 10, 1355--1368. DOI=http://doi.org/10.1109/TVCG.2010.245.

Digital Library

[22]

Tang, A., Owen, C., Biocca, F., and Mou, W. 2003. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florida, USA, April 5--10, 2003). CHI '03. 5, 73--80.

Digital Library

[23]

Schwerdtfeger, B., and Klinker, G. 2008. Supporting order picking with augmented reality. In Proceedings --- 7th IEEE International Symposium on Mixed and Augmented Reality (Cambridge, UK, September 15 -- 18, 2008). ISMAR'08. 91--94. DOI=http://doi.org/10.1109/ISMAR.2008.4637331.

Digital Library

[24]

Funk, M., Kosch, T., and Schmidt, A. 2016. Interactive Worker Assistance: Comparing the Effects of In-Situ Projection, Head-Mounted Displays, Tablet, and Paper Instructions. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (Heidelberg, Germany, 2016), 0--5. DOI=http://doi.org/10.1145/2971648.2971706.

Digital Library

[25]

Grubert, J., Hamacher, D., Mecke, R., Böckelmann, I., Schega, L., Huckauf, A., Urbina, M., Schenk, M., Doil, F., and Tümler, J. 2010. Extended investigations of user-related issues in mobile industrial AR. In Proceedings of the 9th IEEE International Symposium on Mixed and Augmented Reality 2010: Science and Technology (Seoul, Korea, October 13--16). ISMAR'10. 229--230. DOI=http://doi.org/10.1109/ISMAR.2010.5643581.

[26]

Re, G. M., and Bordegoni, M. 2014. An augmented reality framework for supporting and monitoring operators during maintenance tasks. Virtual, Augment. Mix. Real. 443--454. DOI=http://doi.org/10.1007/978-3-319-07464-1_41.

[27]

Ferrise, F., Caruso, G., and Bordegoni, M. 2013. Multimodal training and tele-assistance systems for the maintenance of industrial products. Virtual Phys. Prototyp. 8, 2, 113--126. DOI=http://doi.org/10.1080/17452759.2013.798764.

[28]

Capodieci, A., Mainetti, L., and Alem, L. 2015. An Innovative Approach to Digital Engineering Services Delivery: An Application in Maintenance. 2015 11th Int. Conf. Innov. Inf. Technol. 336--343.

Digital Library

[29]

Funk, M., and Korn, O. 2014. Assisitive Augmentation at the Manual Assembly Workplace using In-Situ Projection. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Toronto, Canada, April 26--May 1,2014). CHI'14.

[30]

Ziegler, J., Heinze, S., and Urbas, L. 2015. The potential of smartwatches to support mobile industrial maintenance tasks. IEEE Int. Conf. Emerg. Technol. Fact. Autom. ETFA. 2015--Octob. DOI=http://doi.org/10.1109/ETFA.2015.7301479.

[31]

Zheng, X. S., Silva, P. M. da, Foucault, C., Dasari, S., Yuan, M., and Goose, S. 2015. Wearable Solution for Industrial Maintenance. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (Seoul, Korea, April 18--23, 2015). CHI'15. 311--314. DOI=http://doi.org/10.1145/2702613.2725442.

Digital Library

[32]

Zheng, X. S., Foucault, C., Silva, P. M. da, Dasari, S., Yang, T., and Goose, S. 2015. Eye-Wearable Technology for Machine Maintenance: Effects of Display Position and Hands-free Operation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Korea, April 18--23, 2015). CHI'15. 1, 2125--2134. DOI=http://doi.org/10.1145/2702123.2702305.

Digital Library

[33]

Grubert, J., Kranz, M., and Quigley, A. 2015. Design and Technology Challenges for Body Proximate Display Ecosystems. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (Copenhagen, Denmark, August 24--27, 2015). MobileHCI'15. 951--954. DOI=http://doi.org/10.1145/2786567.2794310.

Digital Library

[34]

Quigley, A., and Grubert, J. 2015. Perceptual and Social Challenges in Body Proximate Display Ecosystems. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (Copenhagen, Denmark, August 24--27, 2015). MobileHCI'15. 1168--1174. DOI=http://doi.org/10.1145/2786567.2794349.

Digital Library

[35]

Nicolai, T., Sindt, T., Witt, H., Reimerdes, J., and Kenn, H. 2006. Wearable computing for aircraft maintenance: Simplifying the user interface. Proceedings of 3rd Int. Forum Appl. Wearable Comput. (IFAWC). 15 -- 16.

[36]

Witt, H., Nicolai, T., and Kenn, H. 2006. Designing a wearable user interface for hands-free interaction in maintenance applications. In Proceedings of the 4th Annual IEEE International Conference on Pervasive Computing and Communications Workshop-PerCom Workshop (Pisa, Italy, March 13--17, 2006). 652--655. DOI=http://doi.org/10.1109/PERCOMW.2006.39.

Digital Library

[37]

Nicolai, T., Sindt, T., Kenn, H., and Witt, H. 2006. Case Study of Wearable Computing for Aircraft Maintenance. In 3rd International Forum on Applied Wearable Computing (2006), 1--12.

[38]

Navab, N. 2004. Developing killer apps for industrial augmented reality. IEEE Comput. Graph. Appl. 24, 3, 16--20. DOI=http://doi.org/10.1109/MCG.2004.1297006.

Digital Library

[39]

Siegel, J., and Bauer, M. 1997. A field usability evaluation of a wearable system. Dig. Pap. First Int. Symp. Wearable Comput. 1, 18--22. DOI=http://doi.org/10.1109/ISWC.1997.629914.

Digital Library

[40]

Mynttinen, R. 2010. Evaluating spatially directing cues on a wearable user interface in a field setup. Doctoral dissertation. Uniwien.

[41]

Ward, J. A., Lukowicz, P., Tröster, G., and Starner, T. E. 2006. Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans. Pattern Anal. Mach. Intell. 28, 10, 1553--1566. DOI=http://doi.org/10.1109/TPAMI.2006.197.

Digital Library

[42]

Brooke J. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189.

[43]

Davis, F. D. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of. Inf. Technol. MIS Q. 13, 3, 319--340.

Digital Library

[44]

Rashid, U., Nacenta, M. A., and Quigley, A. 2012. The Cost of Display Switching: A Comparison of Mobile, Large Display and Hybrid UI Configurations. In Proceedings of the International Working Conference on Advanced Visual Interfaces (Capri Island, Italy, May 22--25, 2012). AVI'12. 99--106. DOI=http://doi.org/10.1145/2254556.2254577.

Digital Library

[45]

Funk, M., Dingler, T., Cooper, J., and Schmidt, A. 2015. Stop Helping Me --- I'M Bored!: Why Assembly Assistance Needs to Be Adaptive. In Adjunct Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the ACM International Symposium on Wearable Computers (Osaka, Japan, September 7--11, 2015). UbiComp'15. 1269--1273. DOI=http://doi.org/10.1145/2800835.2807942.

Digital Library

[46]

Ronkainen, S., Koskinen, E., Liu, Y., and Korhonen, P. 2010. Environment Analysis as a Basis for Designing Multimodal and Multidevice User Interfaces. Human-Computer Interact. 25, 2, 148--193. DOI=http://doi.org/10.1080/07370020903586712.

[47]

Amft, O., Lauffer, M., Ossevoort, S., Macaluso, F., Lukowicz, P., and Tr, G. 2004. Design of the QBIC wearable computing platform. In Application-Specific Systems, Architectures and Processors, 15th IEEE International Conference on. IEEE (2004), 398--410.

Digital Library

[48]

Gemperle, F., Kasabach, C., Stivoric, J., Bauer, M., and Martin, R. 1998. Design for wearability. Dig. Pap. Second Int. Symp. Wearable Comput. (Cat. No.98EX215). 116--122. DOI=http://doi.org/10.1109/ISWC.1998.729537.

Digital Library

[49]

Gemperle, F., Ota, N., and Siewiorek, D. 2001. Design of a wearable tactile display. In Proceedings of the 5^th International Symposium on Wearable Computers (Zurich, Switzerland, October 8--9, 2001). ISWC'01. 5--12. DOI=http://doi.org/10.1109/ISWC.2001.962082.

Digital Library

[50]

Knight, J. F., Deen-Williams, D., Arvanitis, T. N., Baber, C., Sotiriou, S., Anastopoulou, S., and Gargalakos, M. 2006. Assessing the wearability of wearable computers. In Proceedings of the International Symposium on Wearable Computers (Montreux, Switzerland, October 11--14, 2006). ISWC'06. 75--82. DOI=http://doi.org/10.1109/ISWC.2006.286347.

[51]

Dünser, A., Grasset, R., Seichter, H., and Billinghurst, M. 2007. Applying HCI Principles to AR Systems Design. In Proceedings of 2nd International Workshop on Mixed Reality User Interfaces: Specification, Authoring, Adaptation (North Carolina, USA, March 11, 2007). MRUI '07.

[52]

Billinghurst, M., and Grasset, R. 2005. Designing Augmented Reality Interfaces Physical Elements Input Interaction Metaphor Display Elements Output. ACM Siggraph Comput. Graph. 39, 1, 17--22.

Digital Library

[53]

Kourouthanassis, P. E., Boletsis, C., and Lekakos, G. 2013. Demystifying the design of mobile augmented reality applications. Multimed. Tools Appl. 74, 3, 1045--1066. DOI=http://doi.org/10.1007/s11042-013-1710-7.

Digital Library

Cited By

Heikkilä PAromaa SLammi H(2024)Designing a Tool to Collect Worker Feedback on Novel Solutions in Industrial Work ContextProceedings of the European Conference on Cognitive Ergonomics 202410.1145/3673805.3673821(1-7)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3673805.3673821
Gomes CGuedes TEsteves A(2023)A First Exploration on the Use of Head-Mounted Augmented Reality in the Context of the Portuguese MilitaryProceedings of the ACM on Human-Computer Interaction10.1145/36042397:MHCI(1-11)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604239
Schein KRauschnabel P(2023)Augmented Reality in Manufacturing: Exploring Workers’ Perceptions of BarriersIEEE Transactions on Engineering Management10.1109/TEM.2021.309383370:10(3344-3357)Online publication date: Oct-2023
https://doi.org/10.1109/TEM.2021.3093833
Show More Cited By

Index Terms

Use of wearable and augmented reality technologies in industrial maintenance work
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. Field studies
    2. Interaction paradigms
      1. Mixed / augmented reality
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
    2. Ubiquitous and mobile devices

Recommendations

Industrial maintenance with augmented reality: two case studies
VRST '16: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology

Remote maintenance of industrial manipulators often is performed via telephone support. Recent approaches in the context of the 'Industry 4.0' consider internet technologies and Augmented Reality (AR) to enhance situation awareness between external ...
Condition-based maintenance and production speed optimization under limited maintenance capacity
Abstract
Many industrial facilities consist of multiple units working in parallel to reach a production target or to maximize production revenues. These units deteriorate due to stress that is often dependent on the production speed, and thus maintenance ...
Highlights
- Simultaneous optimization of condition-based maintenance and production speed.
- Maintenance takes considerable time; at most one unit can be maintained at a time.
- Deterioration levels of units are desynchronized when there is a ...
Ethnographic study of a commercially available augmented reality HMD app for industry work instruction
PETRA '19: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

Industrial applications of Augmented Reality (AR) are becoming increasingly commonplace but there are only a small number of published user studies examining the use of commercially available AR technologies, like AR HMDs, with real workers in real ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AcademicMindtrek '16: Proceedings of the 20th International Academic Mindtrek Conference

October 2016

483 pages

ISBN:9781450343671

DOI:10.1145/2994310

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

FIMECC, Finnish Metals and Engineering Competence Cluster

Conference

AcademicMindtrek'16

AcademicMindtrek'16: Academic Mindtrek Conference 2016

October 17 - 18, 2016

Tampere, Finland

Acceptance Rates

Overall Acceptance Rate 110 of 207 submissions, 53%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
1,305
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)6

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Heikkilä PAromaa SLammi H(2024)Designing a Tool to Collect Worker Feedback on Novel Solutions in Industrial Work ContextProceedings of the European Conference on Cognitive Ergonomics 202410.1145/3673805.3673821(1-7)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3673805.3673821
Gomes CGuedes TEsteves A(2023)A First Exploration on the Use of Head-Mounted Augmented Reality in the Context of the Portuguese MilitaryProceedings of the ACM on Human-Computer Interaction10.1145/36042397:MHCI(1-11)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604239
Schein KRauschnabel P(2023)Augmented Reality in Manufacturing: Exploring Workers’ Perceptions of BarriersIEEE Transactions on Engineering Management10.1109/TEM.2021.309383370:10(3344-3357)Online publication date: Oct-2023
https://doi.org/10.1109/TEM.2021.3093833
Nizar AMiwa TUchida M(2023)Examining situational awareness, trust in automation, and workload in engine resources management: an evaluation of head-worn display technologyWMU Journal of Maritime Affairs10.1007/s13437-023-00319-023:1(115-128)Online publication date: 5-Jun-2023
https://doi.org/10.1007/s13437-023-00319-0
Walczak NSobiech FBuczek AJeanty MKupiński KChaniecki ZRomanowski AGrudzień K(2023)Towards Gestural Interaction with 3D Industrial Measurement Data Using HMD ARDigital Interaction and Machine Intelligence10.1007/978-3-031-37649-8_21(213-221)Online publication date: 25-Jul-2023
https://doi.org/10.1007/978-3-031-37649-8_21
Bongiovanni ADe Luca AGava LGrassi LLagomarsino MLapolla MMarino ARoncagliolo PMacciò SCarfì AMastrogiovanni F(2023)Gestural and Touchscreen Interaction for Human-Robot Collaboration: A Comparative StudyIntelligent Autonomous Systems 1710.1007/978-3-031-22216-0_9(122-138)Online publication date: 18-Jan-2023
https://doi.org/10.1007/978-3-031-22216-0_9
Ghobadi MShirowzhan SGhiai MMohammad Ebrahimzadeh FTahmasebinia F(2022)Augmented Reality Applications in Education and Examining Key Factors Affecting the Users’ BehaviorsEducation Sciences10.3390/educsci1301001013:1(10)Online publication date: 22-Dec-2022
https://doi.org/10.3390/educsci13010010
Müller SBaldauf MSeeliger A(2022)Ubiquitous Machinery Monitoring - A Field Study on Manufacturing Workers' User Experience of Mobile and Wearable Monitoring AppsProceedings of the ACM on Human-Computer Interaction10.1145/35467336:MHCI(1-22)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1145/3546733
Buettner RBreitenbach JWannenwetsch KOstermann IPriel R(2022)A Systematic Literature Review of Virtual and Augmented Reality Applications for Maintenance in Manufacturing2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC54236.2022.00099(545-552)Online publication date: Jun-2022
https://doi.org/10.1109/COMPSAC54236.2022.00099
Zhang XHe WBillinghurst MLiu DYang LFeng SLiu Y(2022)Usability of Cross-Device Interaction Interfaces for Augmented Reality in Physical TasksInternational Journal of Human–Computer Interaction10.1080/10447318.2022.216053740:9(2361-2379)Online publication date: 28-Dec-2022
https://doi.org/10.1080/10447318.2022.2160537
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents