1 s2.0 S2212827121001542 Main
1 s2.0 S2212827121001542 Main
1 s2.0 S2212827121001542 Main
com
ScienceDirect
Procedia CIRP 98 (2021) 400–405
www.elsevier.com/locate/procedia
Abstract
Augmented Reality Technology Augmented Reality Technology (AR) is one of the key technologies of digital transformation in industrial and
non-industrial areas. Due to the rapid development of display hardware and tracking systems, virtual and augmented reality applications are being
developed today that would only have been possible in large research laboratories with huge efforts a few years ago. This paper presents the
application of the augmented reality technology in the industrial field. Based on industrial case studies the implementation of two AR applications
and their potential to digitize the product lifecycle processes will be discussed. The case studies were developed within cooperation projects
between the University of Applied Sciences Karlsruhe and a fire truck manufacturer with the focus on the following topics: product configuration
management and assembly assistance. The objective of this work is the investigation of the AR application in the industrial environment and its
capability as digital transformation technology along the product lifecycle. The presented paper delivers experiences and suggestions in terms of
praxis-oriented development and deployment of the AR technology in the engineering field, which can be used for future AR research works.
© 2021 The Authors. Published by Elsevier B.V.
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0)
Peer-review under responsibility of the scientific committee of the 28th CIRP Conference on Life Cycle Engineering.
Keywords: Augmented Reality (AR); Product Configuration Management; Production Planning and Assembly Assistance; Idustrial Digitization.
This is a resupply of March 2023 as the template used in the publication of the original article contained errors. The content of the article has remained unaffected.
Fahmi Bellalouna / Procedia CIRP 98 (2021) 400–405 401
This is a resupply of March 2023 as the template used in the publication of the original article contained errors. The content of the article has remained unaffected.
402 Fahmi Bellalouna / Procedia CIRP 98 (2021) 400–405
summary about the significant works is given without claim of complex knowledge work, with an initial emphasis on software
completeness. development, although it has been used in other fields and is
Quandt et al. [13] discussed and defined the general slowly starting to be explored for other complex work, research
requirements for AR application based on two industrial case and advanced technologies [19]. Scrum is suitable for
studies: an AR-based assistance system for wind energy development projects dealing with new innovative
services technicians and an AR-based welding simulator for technologies with unpredictable challenges such as the AR
education and training purposes. In both case studies, the technology, for which conventional predictive project
authors identified approaches to meet the defined requirements management approaches are not suited [19]. For the
for successful application of AR solutions in industrial implementation of the case studies a so-called scrum team was
scenarios. The work of Quandt et al. only presented a system built, which contains a representative from the business unit
architecture for AR applications and does not provide any “product owner”, a “development team” of students and a
information about their implementation and their introduction project manager “scrum master”. The implementation of the
in the industrial practice. Weiding et al. [15] presented a new case studies was structured in several successive iteration loops
approach for a systematic development process for mobile AR also known as sprint. Each sprint starts with a sprint planning
application to support production planning. The developed event that establishes a sprint goal. The team accepts what they
approach delivers a theoretic basic methodology how to design agree is ready and translate this into a sprint backlog, with a
an AR application for a production planning process, however breakdown of the work required and an estimated forecast for
it has not been realized into a concrete AR application and there the sprint goal. Each sprint ends with a sprint review and sprint
is a lack of how to be applied in the practice. Mura et al. [16] retrospective, that reviews progress to show to stakeholders and
proposed an approach for system based on the interaction identify lessons and improvements for the next sprints [20].
between a force sensor and an AR equipment, used to give the In order to evaluate the implemented AR applications
worker during a manual assembly process the necessary systematically, a questionnaire was created and used as basis
information about the correct assembly sequence and alert him for the evaluation workshops with the business unit experts and
in case of errors. During this work two AR devices were used key users. The evaluation workshops were structured in three
and tested: a video-mixing spatial display and an optical see- phases: introduction phase to present the AR application and its
through apparatus. The presented approach was developed for functions (time duration about 10 min), field trial phase with
a small and simple assembly process and had prototypical the experts to test the AR application in real field situations
character with open issues and weaknesses. Servan et al. [17] (time duration about 30 min), and the assessment phase to fill
developed an approach for using AR in AIRBUS A400M shop out the questionnaire (time duration about 30 min). The
floor assembly work instructions. To replace the elaborate questionnaire focused on the following topics:
paper-based work instructions (e.g. drawings of components, Performance: the ability of the AR application to exact
torques to be applied, sealing system characteristics or paste, overlay of the virtual data on the physical object and the
etc.) AIRBUS Military developed within the MOON project real-time rendering when changing user position.
(asseMbly Oriented authOring augmeNted reality) an AR- Usability: intuitive use of the AR functions without any
based assembly work instruction approach. MOON uses 3D hindrance while real task operation.
information from the iDMU (industrial Digital Mock-Up) to Added value: the perception of added value for the
generate assembly instructions by applying Augmented Reality considered business processes due to the use of the AR
technology. A demonstrator was developed for the electrical application.
harness routing in the of the AIRBUS A400M. The results of The outcomes of the final assessment of the questionnaire was
the project showed that the use of augmented reality in the shop a list of recommendation for the involved business units on how
floor was a feasible alternative to the conventional paper-based to deal with the AR technology in the future in terms of aptitude
documentation. The project proposed a methodology, named for the own business processes and the application scope.
MOON, based on the combination of these two techniques:
iDMU and augmented reality. The paper of Porcelli et al. [18] 5. Case Study: AR-based product configuration
presents a case study in an industrial context, that depicts the management for rescue platform
main technical and organizational challenges that have to be
faced in order to introduce AR in the delivery process of The rescue platform of a fire engine has approx. 50 different
maintenance services. Within this case study the technical and special and optional equipment (e.g. diver leader, stretcher,
organizational issues that can limit the actual introduction of wheelchair ramp, camera, lighting) that can be selected by the
this technology were identified and discussed. The authors customer while the product configuration. The product
believe the AR technology is directed, within a few years, to configuration is currently carried out with the help of the sales
become a standard equipment of service and maintenance documents, which exist in paper form as product catalogs. The
department. product catalogs are created individually for the single
equipment and provide a functional description with a simple
4. Methodology image presentation of the assembly position. In this way, only
one optional equipment can be considered at the same time, so
The realization of the presented case studies has been done that the space available when installing two neighboring
applying the scrum product design and project management components is not apparent. It also remains unclear whether the
framework. Scrum is an agile process framework for managing components may collide and a simultaneous installation is not
This is a resupply of March 2023 as the template used in the publication of the original article contained errors. The content of the article has remained unaffected.
Fahmi Bellalouna / Procedia CIRP 98 (2021) 400–405 403
possible. In addition, the installation position can only be Load of technical documents function loads and opens
viewed from a fixed perspective. This current configuration technical documents as PDF for the built equipment. Hence,
process of the rescue platform leads in the most of the case to the customer does not have to page through the product manual
wrong decision regarding the selection of the optional to look for the technical specification of a specific components.
equipment, which not fit to each other. However, these wrong As consequence, a valuable time during the configuration
decisions can only be detected during the assembly process of process can be saved and mistakes in terms of using of incorrect
the rescue platform. As consequence is cost- and time- data can be avoided.
consuming reconfiguration of the rescue platform, which leads Play of videos function loads and plays videos to
to costumer dissatisfaction and in some cases to customer demonstrate the use of the selected equipment in the field
churning. In order to simplify the configuration process for the during a rescue operation. This function offers a crucial support
rescue platform, an AR application was developed in this for the costumer while the decision process. In this case, the
cooperation project between the University of Applied configuration process will be carried out based on accurate
Sciences Karlsruhe and an OEM for fire trucks. Using the AR technical information and not on assumptions.
application, the costumer can overlay the virtual 3D models of
all the optional and extra equipment to a real basic rescue
platform. To use the AR application, only a real rescue platform
with basic equipment without additional components is
required. The assembly, disassembly and transport of special
equipment for demonstrative purposes at sales events is
omitted, which greatly simplifies and speeds up the
configuration process. Below the main function of the AR
application are presented:
Model recognition and tracking function uses a 2D
drawing as target model to orient the camera of the AR device
to real basic rescue platform and to detect it (Fig. 2). Once the Fig. 3. AR function “Overlay 3D model for optional equipment”
model is detected, the virtual 3D model of the rescue platform
is overlaid on the physical real one. This function allows an 6. Case Study: AR-based assembly assistance for
uninterrupted model tracking even though the customer move firefighting equipment
around. The detection of the rescue platform based on catalog
image can also be attained by this function. The firefighting equipment (e.g. power saw, axe, fire truck,
hoses) is housed in various equipment boxes in a firetruck.
There is a wide variety of equipment configurations that can be
stored in the different equipment cases with different positions
(Fig. 4). The equipment configuration is determined in
collaboration with the fire department during the sales process
individually. Currently, the assembly process of the firefighting
equipment is based on paper 2D drawings, which depict the
equipment and its location in the firetruck. However, this
process is very elaborate and error-prone due to the high variety
Fig. 2. AR function “Model recognition” of the equipment, its storage and positioning. This process is
Tree view function structures the optional and extra particularly difficult for inexperienced employees and leads to
equipment into several categories (e.g. safety, extinguish high error rates while the assembly process. This causes high
system, camera system). Hence, the costumer can easy navigate numbers of customer complaints, which lead to high costs for
in the virtual equipment catalog and search for the needed guarantee and after sales service, as well as customer
components systematically. The content of a category expands dissatisfaction.
as sub-tree view by selecting a category in the main tree view. In order to improve the quality and to reduce the error rate
Overlay 3D Model function enables the display the virtual during the assembly process, an AR-based assembly assistant
3D model of one or more selected components in the tree view application was developed in cooperation with the firefighter
based on the real rescue platform in the correct position and equipment manufacturer. The AR application is intended to
size (Fig. 5). This function contains configuration rules that display the firefighting equipment virtually in the empty
prevent the selection of conflicting components. The tracking physical equipment cases using a mobile device (e.g. tablet,
function permits the observation of the shown 3D model from smartphone). This should make it easier for new employees to
different perspective for assembly and collusion check. get started with assembly process and support them during the
Animation function animates the in the tree view selected firefighting equipment installation. In addition, the AR
components. The animation shows the real movement of the application can be used to visualize the final assembly state of
components in the 3D space. Using this function, the costumer the firefighting equipment to the fire department in the earlier
can get a realistic impression of the built equipment and can sales process phase. This leads to avoid misunderstandings
check its collision with the neighboring components. Thus, a during the requirements management process and to identify
reliable and efficient configuration process can be ensured. change requests at an early product planning phase. In
This is a resupply of March 2023 as the template used in the publication of the original article contained errors. The content of the article has remained unaffected.
404 Fahmi Bellalouna / Procedia CIRP 98 (2021) 400–405
following the main functions of the AR application are 7. Implementation of the AR applications
outlined.
The presented AR applications were implemented using the
software systems Vuforia SDK and Unity3D Engine. An
important element of an AR application is the model target. A
model target is a real-world object that has to be recognized in
the AR application by the camera of the AR device. It can be
an image, e.g. a photo or a page in a magazine, or a real object
like a chair. In order to have a stable and reliable detection, the
real object must be static and have a distinctive shape. The
target objects in this work were created with the Vuforia SDK
based on the 3D CAD models of the real objects (Fig. 7). The
UX (user experience) in the AR applications is designed and
Fig. 4. Fire truck with firefighting equipment (Rosenbauer)
implemented in Unity3D Engine. The UX describes the design
To start the AR-based assembly process, the QR code of the
of the UI (user interface) in interactive context. Before the
fire truck case that have to be assembled has to be scanned.
development of the UX, the requirements for the AR
Based on this information the virtual data of the firefighting
application were defined. These were derived from the tasks of
equipment will be loaded while the following steps.
the target user groups. The best method to collect the user tasks
Afterwards, an AR visual target for the concerned equipment
and to identify the user interactions with the AR application,
box will appear, which can serve to identify and track the real
was the discussion with the user groups the typical everyday
box in the fire truck (Fig. 5).
experiences. The final AR applications were built from
Unity3D as Android Apps that can be run on any Android
device (Smartphone, Tablet) that fulfills the hard- and software
requirements.
This is a resupply of March 2023 as the template used in the publication of the original article contained errors. The content of the article has remained unaffected.
Fahmi Bellalouna / Procedia CIRP 98 (2021) 400–405 405
configuration thank the engineers and the employees of Rosenbauer for their
changes can be support in carrying out this work and in evaluating and
avoided. discussing the implemented AR applications.
AR-based Smooth target The AR The overlying of
assembly detection and application the virtual 3D References
assistance for display of the 3D offers a good models on the real
firefighting models of the UX with very equipment box [1] Schellinger J, Tokarski K O, Kissling-Näf I. Digital Transformation und
Unternehmensführung: Trend und Perspektiven für die Praxis, Springer
equipment firefighting interactive, can increase the
Gabler; 2019.
equipment. simple and efficiency and [2] Fitzgerald M, Kruschwitz M, Bonnet D, Welch M. Embracing digital
Due to the one- context- reduce time of the technology: A new strtegic imperative. In: MIT Sloan Management Review
side access to the oriented assembly process. 55(2); 2014.
equipment box, a functions. The use of the AR [3] Dörner R, Broll W, Grimm P, Jung B. Virtual and Augmented Reality
(VR/AR): Grundlagen und Methoden der Virtuellen und Augmentierten
whole 3D Though, the use application can
Realität, Springer Vieweg; 2013.
visualization and of tablet or also compensate [4] Keep Calm and Digitize: What Approach are German Companies Taking
tracking of the smartphone as the qualification in a New Age?, The trend study by Tata Consultancy Services (TCS) and
different models AR device lack of the Bitkom Research. https://studie-digitalisierung.de/en, 2019, visited on 26th
is quite difficult. while the assembly August, 2020.
[5] Azuma R T. A Survey of Augemented Reality, in Teleoperators and Virtual
assembly operators.
Environments 6; 1997. p. 355-385.
process is [6] Augmented Reality or Mixed Reality?. https://www.augmented-
impediment minds.com/en/augmented-reality/what-is-augmented-reality/, visited on
and limits the 26th August, 2020.
activities of the [7] Anouk de Regt A, Barnes S J. V-Commerce in Retail: Nature and Potential
Impact, in Augmented Reality and Virtual Reality: The Power of AR and
assembly
VR Business, Springer; 2019. p. 17-23.
operators. The [8] Bade C, Schreiber W. Einsatz von Augmented Reality in der Fertigung, in
use of HMD Virtual and Augmented Reality (VR/AR): Grundlagen und Methoden der
AR device Virtuellen und Augmentierten Realität, Springer Vieweg; 2013. p. 376-
could improve 378.
[9] Molnár G, Sik D. Smart devices, Samrt environments, samrt students – A
the usability.
review on eductional opportunities in virtual and augmented reality
learning environmnets. In: proceedings of 10th IEEE International
Based on the evaluation feedback a list of actions and Conference on Cognitive Infocommunications – CogInfoCom; 2019. p.
recommendations are being defined at the moment. The 495-498.
findings from the evaluations of the use cases will be used as a [10] Molnár G, Szüts Z, Biró K. Use of Augmented Reality in Learning. In:
Acta Polytechnica Hungarica, Vol 15, No. 5; 2018. p. 209-222.
basis for carrying out future projects. [11] Horvath I, Sudar A. Factors contributing to the enhanced performance of
the maxwhere 3d vr platform in the distribution of digital information. In:
9. Conclusion Acta Polytechnica Hungarica, Vol 15, No. 3; 2018. p. 149-173
[12] Billinghurst M, Clark A, Lee G. Survey of Augmented Reality. In:
Thanks to Augmented Reality Technology, complex Foundations and Trends in Human-Computer Interaction; 2014. p. 73-272.
[13] Quandt M, Knoke B, Gorldt C, Freitag M, Thoben K D. General
technical systems can be represented digitally in a realistic
Requirements for Industrial Augmented Reality Applications. In:
virtual environment. This enables users to completely immerse proceedings of 51st CIRP Conference on Manufacturing Systems; 2018. p.
in the virtual environment and to observe and interact with the 1131-1135.
systems and objects contained therein without major [14] Nee A Y C, Ong S K, Chryssolouris G, Mourtzis D. Augmented reality
restrictions, or to enhance real products and systems with applications in design and manufacturing. In: proceedings CIRP Annals –
Manufacturing Technology; 2012. p. 657-679.
digital data in runtime. This creates new opportunities to
[15] Weidig C, Menck N, Aurich J C. Systematic Development of Mobile AR-
present the behavior and functionalities of complex systems in Applications Supporting Production Planning. In: proceedings of 5th CIRP
a tangible and understandable way. The AR technology thus International Conference on Changeable, Agile, Reconfiguration and
has the ability to revolutionize industrial business processes. Virtual Production; 2013. p. 219-224.
Based on industrial use cases the implementation of two AR [16] Della Mura M, Gino D, Failli F. An integrated environment based on
augmented reality and sensing device for manual assembly workstations.
applications and their potential to digitize the industrial
In: proceedings of 48th CIRP Conference on Manufacturing Systems;
business processes was discussed in this paper. The objective 2015. p. 342-345.
of this work was the investigation of the AR application in the [17] Servá J, Mas F, Menéndez J L, Rios J. Using augmented reality in
industrial environment and its capability as digital AIRBUS A400M shop florr assembly work instructions. In: proceedings of
transformation technology in the industrial field. AIP Conference; 2012. p. 633-640.
[18] Porcelli I, Rapaccini M, Espíndola D B, Pereira C E. Technical and
Organizational Issues about the Introduction of Augmented Reality in
Acknowledgements Maintenance and Technical Assistance Services. In: proceedings of 11th
IFAC Workshop on Intelligent Manufacturing Systems The International
The presented case studies were conducted during the Federation of Automatic Control; 2013. p. 257-262.
cooperation project between the University of applied Sciences [19] Agile 101. In: https://www.agilealliance.org/agile101/, visited on 26th
August, 2020.
Karlsruhe and the Firefighting-Truck manufacturer
[20] Sutherland J, K. Schwaber K. The Scrum Papers: Nut, Bolts, and Origins
Rosenbauer Karlsruhe GmbH & Co. KG. We would like to of an Agile Framework. In: scruminc; 2011.
This is a resupply of March 2023 as the template used in the publication of the original article contained errors. The content of the article has remained unaffected.