research-article

Towards a reference software architecture for human-AI teaming in smart manufacturing

Authors:

Philipp Haindl,

Georg Buchgeher,

Maqbool Khan, and

Bernhard MoserAuthors Info & Claims

ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results

May 2022

Pages 96 - 100

https://doi.org/10.1145/3510455.3512788

Published: 17 October 2022 Publication History

Abstract

With the proliferation of AI-enabled software systems in smart manufacturing, the role of such systems moves away from a reactive to a proactive role that provides context-specific support to manufacturing operators. In the frame of the EU funded Teaming.AI project, we identified the monitoring of teaming aspects in human-AI collaboration, the runtime monitoring and validation of ethical policies, and the support for experimentation with data and machine learning algorithms as the most relevant challenges for human-AI teaming in smart manufacturing. Based on these challenges, we developed a reference software architecture based on knowledge graphs, tracking and scene analysis, and components for relational machine learning with a particular focus on its scalability. Our approach uses knowledge graphs to capture product-and process specific knowledge in the manufacturing process and to utilize it for relational machine learning. This allows for context-specific recommendations for actions in the manufacturing process for the optimization of product quality and the prevention of physical harm. The empirical validation of this software architecture will be conducted in cooperation with three large-scale companies in the automotive, energy systems, and precision machining domain. In this paper we discuss the identified challenges for such a reference software architecture, present its preliminary status, and sketch our further research vision in this project.

References

[1]

Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software Engineering for Machine Learning: A Case Study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 291--300.

Digital Library

[2]

Anders Arpteg, Björn Brinne, Luka Crnkovic-Friis, and Jan Bosch. 2018. Software Engineering Challenges of Deep Learning. In 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). 50--59.

[3]

Jan Bosch, Ivica Crnkovic, and Helena Holmström Olsson. 2020. Engineering AI Systems: A Research Agenda. arXiv:2001.07522 [cs] (June 2020). http://arxiv.org/abs/2001.07522 arXiv: 2001.07522.

[4]

OPC Foundation. 2008. What is OPC? https://opcfoundation.org/about/what-is-opc/ Accessed 2021-10-14.

[5]

Felix Gervits, Terry W. Fong, and Matthias Scheutz. 2018. Shared Mental Models to Support Distributed Human-Robot Teaming in Space. In 2018 AIAA SPACE and Astronautics Forum and Exposition. American Institute of Aeronautics and Astronautics.

[6]

Thilo Hagendorff. 2020. The Ethics of AI Ethics: An Evaluation of Guidelines. Minds and Machines 30, 1 (March 2020), 99--120.

Digital Library

[7]

HLEG. 2019. Ethics guidelines for trustworthy AI | Shaping Europe's digital future. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai Accessed 2021-10-10.

[8]

IEEE. 2019. IEEE Global A/IS Ethics Initiative. https://standards.ieee.org/industry-connections/ec/autonomous-systems.html Accessed 2021-10-10.

[9]

Brittany Johnson and Justin Smith. 2021. Towards Ethical Data-Driven Software: Filling the Gaps in Ethics Research amp; Practice. In 2021 IEEE/ACM 2nd International Workshop on Ethics in Software Engineering Research and Practice (SEthics). 18--25.

[10]

Matthew Johnson and Alonso Vera. 2019. No AI Is an Island: The Case for Teaming Intelligence. AI Magazine 40, 1 (March 2019), 16--28. Number: 1.

Digital Library

[11]

Catholijn M. Jonker, M. Birna van Riemsdijk, and Bas Vermeulen. 2011. Shared Mental Models. In Coordination, Organizations, Institutions, and Norms in Agent Systems VI (Lecture Notes in Computer Science), Marina De Vos, Nicoletta Fornara, Jeremy V. Pitt, and George Vouros (Eds.). Springer, Berlin, Heidelberg, 132--151.

[12]

Ruhul Amin Khalil, Nasir Saeed, Mudassir Masood, Yasaman Moradi Fard, Mohamed-Slim Alouini, and Tareq Y. Al-Naffouri. 2021. Deep Learning in the Industrial Internet of Things: Potentials, Challenges, and Emerging Applications. IEEE Internet of Things Journal 8, 14 (July 2021), 11016--11040. Conference Name: IEEE Internet of Things Journal.

[13]

Franz Georg Listl, Jan Fischer, Dagmar Beyer, and Michael Weyrich. 2020. Knowledge Representation in Modeling and Simulation: A survey for the production and logistic domain. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vol. 1. 1051--1056. ISSN: 1946-0759.

[14]

Jakob Mökander, Jessica Morley, Mariarosaria Taddeo, and Luciano Floridi. 2021. Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations. Science and Engineering Ethics 27, 4 (July 2021), 44.

[15]

Anh Nguyen-Duc and Pekka Abrahamsson. 2020. Continuous experimentation on artificial intelligence software: a research agenda. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1513--1516.

Digital Library

[16]

Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2016. A Review of Relational Machine Learning for Knowledge Graphs. Proc. IEEE 104, 1 (Jan. 2016), 11--33. Conference Name: Proceedings of the IEEE.

[17]

Sudip Phuyal, Diwakar Bista, and Rabindra Bista. 2020. Challenges, Opportunities and Future Directions of Smart Manufacturing: A State of Art Review. Sustainable Futures 2 (2020), 100023.

[18]

S Pichai. 2018. AI at Google: our principles. https://blog.google/technology/ai/ai-principles/ Accessed 2021-10-10.

[19]

Davy Preuveneers, Yolande Berbers, and Wouter Joosen. 2016. SAMURAI: A batch and streaming context architecture for large-scale intelligent applications and environments. Journal of Ambient Intelligence and Smart Environments 8, 1 (Jan. 2016), 63--78. Publisher: IOS Press.

Digital Library

[20]

Y. J. Qu, X. G. Ming, Z. W. Liu, X. Y. Zhang, and Z. T. Hou. 2019. Smart manufacturing systems: state of the art and future trends. The International Journal of Advanced Manufacturing Technology 103, 9 (Aug. 2019), 3751--3768.

[21]

Martin Ringsquandl, E. Kharlamov, Daria Stepanova, Marcel Hildebrandt, S. Lamparter, R. Lepratti, I. Horrocks, and Peer Kröger. 2018. Filling Gaps in Industrial Knowledge Graphs via Event-Enhanced Embedding. In Proceedings of ISWC 2018 Posters & Demonstrations, Industry and Blue Sky Ideas Tracks co-located with 17th International Semantic Web Conference (ISWC 2018). CEUR Workshop Proceedings (CEUR-WS.org). http://ceur-ws.org/Vol-2180/paper-52.pdf

[22]

Martin Ringsquandl, Evgeny Kharlamov, Daria Stepanova, Steffen Lamparter, Raffaello Lepratti, Ian Horrocks, and Peer Kröger. 2017. On event-driven knowledge graph completion in digital factories. In 2017 IEEE International Conference on Big Data (Big Data). 1676--1681.

[23]

Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (May 2019), 206--215. Bandiera_abtest: a Cg_type: Nature Research Journals Number: 5 Primary_atype: Reviews Publisher: Nature Publishing Group Subject_term: Computer science;Criminology;Science, technology and society;Statistics Subject_term_id:computer-science;criminology;science-technology-and-society;statistics.

[24]

Eduardo Salas, Dana E. Sims, and C. Shawn Burke. 2005. Is there a "Big Five" in Teamwork? Small Group Research 36, 5 (Oct. 2005), 555--599. Publisher: SAGE Publications Inc.

[25]

Nuno Santos, Francisco Morais, Helena Rodrigues, and Ricardo J. Machado. 2019. Systems Development for the Industrial IoT: Challenges from Industry R&D Projects. In The Internet of Things in the Industrial Sector: Security and Device Connectivity, Smart Environments, and Industry 4.0, Zaigham Mahmood (Ed.). Springer International Publishing, Cham, 55--78.

[26]

Matthias Scheutz, Scott A. DeLoach, and Julie A. Adams. 2017. A Framework for Developing and Using Shared Mental Models in Human-Agent Teams. Journal of Cognitive Engineering and Decision Making 11, 3 (Sept. 2017), 203--224. Publisher: SAGE Publications.

[27]

Julien Siebert, Lisa Jöckel, Jens Heidrich, Koji Nakamichi, Kyoko Ohashi, Isao Namba, Rieko Yamamoto, and Mikio Aoyama. 2020. Towards Guidelines for Assessing Qualities of Machine Learning Systems. In Quality of Information and Communications Technology - 13th International Conference, QUATIC 2020, Faro, Portugal, September 9-11, 2020, Proceedings (Communications in Computer and Information Science, Vol. 1266), Martin J. Shepperd, Fernando Brito e Abreu, Alberto Rodrigues da Silva, and Ricardo Pérez-Castillo (Eds.). Springer, 17--31.

[28]

Théo Trouillon, Christopher R. Dance, Éric Gaussier, Johannes Welbl, Sebastian Riedel, and Guillaume Bouchard. 2017. Knowledge graph completion via complex tensor factorization. The Journal of Machine Learning Research 18, 1 (Jan. 2017), 4735--4772.

[29]

Ville Vakkuri, Kai-Kristian Kemell, and Pekka Abrahamsson. 2019. Ethically Aligned Design: An Empirical Evaluation of the RESOLVEDD-Strategy in Software and Systems Development Context. In 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). 46--50.

[30]

Ville Vakkuri, Kai-Kristian Kemell, Marianna Jantunen, Erika Halme, and Pekka Abrahamsson. 2021. ECCOLA --- A method for implementing ethically aligned AI systems. Journal of Systems and Software 182 (Dec. 2021), 111067.

Digital Library

[31]

James Warren and Nathan Marz. 2015. Big Data: Principles and best practices of scalable realtime data systems (1st edition ed.). Manning.

Cited By

Mohan JRanganathan PReza H(2024)Hierarchical Software Framework for Safe Unmanned Aerial Systems Integration into National Airspace (NAS)2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC60891.2024.10427776(0198-0206)Online publication date: 8-Jan-2024
https://doi.org/10.1109/CCWC60891.2024.10427776
Rajesh ASharma AChinthaginjala RPau GAbbas MShankar TFouziya Sulthana S(2024)Enhancing visual clarity in rainy conditions based on single-frame filtering algorithmAin Shams Engineering Journal10.1016/j.asej.2024.102846(102846)Online publication date: May-2024
https://doi.org/10.1016/j.asej.2024.102846
Hoch TMartinez-Gil JPichler MSilvina AHeinzl BMoser BEleftheriou DEstrada-Lugo HLeva M(2023)Multi-Stakeholder Perspective on Human-AI Collaboration in Industry 5.0Artificial Intelligence in Manufacturing10.1007/978-3-031-46452-2_23(407-421)Online publication date: 28-Sep-2023
https://doi.org/10.1007/978-3-031-46452-2_23
Show More Cited By

Index Terms

Towards a reference software architecture for human-AI teaming in smart manufacturing

Recommendations

IoT Real-Time Production Monitoring and Automated Process Transformation in Smart Manufacturing

Conventional automobile manufacturing plants involve intricate assembly, testing, and debugging processes heavily reliant on manual operations. This study aims to explore the application of industrial internet of things (IIoT) and deep learning ...
Read More
Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues
Highlights
This paper reviews the current status and advancement of Digital Twin-driven smart manufacturing, with highlights on the following aspects:
- Presented the connotation of ...
Abstract
This paper reviews the recent development of Digital Twin technologies in manufacturing systems and processes, to analyze the connotation, application scenarios, and research issues of Digital Twin-driven smart manufacturing in the ...
Read More
Editorial: intelligent manufacturing systems towards industry 4.0 era
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results

May 2022

143 pages

ISBN:9781450392242

DOI:10.1145/3510455

Conference Chairs:
Liliana Pasquale
University College Dublin and Lero, Ireland
,
Christoph Treude
University of Melbourne, Australia

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

EU Horizon 2020

Conference

ICSE '22

Sponsor:

SIGSOFT

ICSE '22: 44th International Conference on Software Engineering

May 21 - 29, 2022

Pennsylvania, Pittsburgh

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
151
Total Downloads

Downloads (Last 12 months)86
Downloads (Last 6 weeks)8

Other Metrics

View Author Metrics

Citations

Cited By

Mohan JRanganathan PReza H(2024)Hierarchical Software Framework for Safe Unmanned Aerial Systems Integration into National Airspace (NAS)2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC60891.2024.10427776(0198-0206)Online publication date: 8-Jan-2024
https://doi.org/10.1109/CCWC60891.2024.10427776
Rajesh ASharma AChinthaginjala RPau GAbbas MShankar TFouziya Sulthana S(2024)Enhancing visual clarity in rainy conditions based on single-frame filtering algorithmAin Shams Engineering Journal10.1016/j.asej.2024.102846(102846)Online publication date: May-2024
https://doi.org/10.1016/j.asej.2024.102846
Hoch TMartinez-Gil JPichler MSilvina AHeinzl BMoser BEleftheriou DEstrada-Lugo HLeva M(2023)Multi-Stakeholder Perspective on Human-AI Collaboration in Industry 5.0Artificial Intelligence in Manufacturing10.1007/978-3-031-46452-2_23(407-421)Online publication date: 28-Sep-2023
https://doi.org/10.1007/978-3-031-46452-2_23
Sathish KAnbazhagan RVenkata RArena FPau G(2022)Investigation and Numerical Simulation of the Acoustic Target Strength of the Underwater Submarine VehicleInventions10.3390/inventions70401117:4(111)Online publication date: 1-Dec-2022
https://doi.org/10.3390/inventions7040111
Khan MAhmad ASobieczky FPichler MMoser BBukovsky I(2022)A Systematic Mapping Study of Predictive Maintenance in SMEsIEEE Access10.1109/ACCESS.2022.320069410(88738-88749)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3200694

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents