Abstract
Human-robot teaming is crucial for future automation in small and medium enterprises. In that context, domain-specific process models are used as an intuitive description of work to share between two agents. Process designers usually introduce a certain degree of abstraction into the models. This way, models are better to trace for humans, and a single model can moreover enable flexibility by capturing several process variations. However, abstraction can lead to unintentional omission of information (e.g., experience of skilled workers). This may impair the quality of process results. To balance the trade-off between model readability and flexibility, we contribute a novel human-robot teaming approach with incremental learning of relevant process details (RPDs). RPDs are extracted from imagery during process execution and used to enrich an integrated process model which unifies human worker instruction and robot programming. Experiments based on two use cases demonstrate the practical feasibility and scalability of our approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
https://www.tensorflow.org/ (Accessed: 02 May 2023).
- 2.
https://github.com/marcotcr/lime (Accessed: 30 April 2023).
- 3.
https://scikit-image.org/docs/dev/api/skimage.segmentation.html (Accessed: 08 May 2023).
References
Ahmadikatouli, A., Aboutalebi, M.: New evolutionary approach to business process model optimization. In: Proceedings of the International MultiConference of Engineers and Computer Scientists, vol. 2 (2011)
Andersen, R.H., Solund, T., Hallam, J.: Definition and initial case-based evaluation of hardware-independent robot skills for industrial robotic co-workers. In: 41st International Symposium on Robotics (ISR), pp. 1–7. VDE (2014)
Becker, J., Rosemann, M., Von Uthmann, C.: Guidelines of business process modeling. In: BPM: Models, Techniques, and Empirical Studies, pp. 30–49 (2002)
Beumelburg, K.: Fähigkeitsorientierte Montageablaufplanung in der direkten Mensch-Roboter-Kooperation (2005)
Bobrik, R., Reichert, M., Bauer, T.: View-based process visualization. In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 88–95. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75183-0_7
Chinosi, M., Trombetta, A.: BPMN: an introduction to the standard. Comput. Stand. Interfaces 34(1), 124–134 (2012)
Coradeschi, S., Saffiotti, A.: An introduction to the anchoring problem. Robot. Auton. Syst. 43(2–3), 85–96 (2003)
Darvish, K., et al.: A hierarchical architecture for human-robot cooperation processes. IEEE Trans. Rob. 37(2), 567–586 (2021)
Dietz, T., et al.: Programming system for efficient use of industrial robots for deburring in SME environments. In: 7th German Conference on Robotics, pp. 1–6 (2012)
Fichtner, M., Fichtner, U.A., Jablonski, S.: An experimental study of intuitive representations of process task annotations. In: Sellami, M., Ceravolo, P., Reijers, H.A., Gaaloul, W., Panetto, H. (eds.) CoopIS 2022. LNCS, vol. 13591, pp. 311–321. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-17834-4_19
Fichtner., M., Schönig., S., Jablonski., S.: How lime explanation models can be used to extend business process models by relevant process details. In: Proceedings of the 24th International Conference on Enterprise Information Systems. vol. 2, pp. 527–534 (2022)
Gounaris, A.: Towards automated performance optimization of BPMN business processes. In: New Trends in Database and Information Systems (ADBIS), pp. 19–28 (2016)
Hasegawa, T., Suehiro, T., Takase, K.: A model-based manipulation system with skill-based execution. IEEE Trans. Robot. Autom. 8(5), 535–544 (1992)
Koschmider, A., et al.: Business process modeling support by depictive and descriptive diagrams. In: Enterprise Modelling and Information Systems Architectures, pp. 31–44 (2015)
Matheson, E., Minto, R., Zampieri, E.G., Faccio, M., Rosati, G.: Human-robot collaboration in manufacturing applications: a review. Robotics 8(4), 100 (2019)
Mendling, J., Reijers, H.A., van der Aalst, W.M.: Seven process modeling guidelines (7pmg). Inf. Softw. Technol. 52(2), 127–136 (2010)
Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)
Niedermann, F., Radeschütz, S., Mitschang, B.: Deep business optimization: a platform for automated process optimization. INFORMATIK - Business Process and Service Science - Proceedings of ISSS and BPSC (2010)
Nottensteiner, K., et al.: A complete automated chain for flexible assembly using recognition, planning and sensor-based execution. In: Proceedings of 47st International Symposium on Robotics (ISR), pp. 1–8 (2016)
Paxton, C., et al.: Costar: instructing collaborative robots with behavior trees and vision. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 564–571 (2017)
Pedersen, M.R., et al.: Robot skills for manufacturing: from concept to industrial deployment. Robot. Comput. Integr. Manuf. 37, 282–291 (2016)
Polyvyanyy, A., Smirnov, S., Weske, M.: Process model abstraction: a slider approach. In: 12th International IEEE Enterprise Distributed Object Computing Conference (2008)
Reichert, M., et al.: Enabling personalized visualization of large business processes through parameterizable views. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp. 1653–1660 (2012)
Ribeiro, M.T., Singh, S., Guestrin, C.: “why should i trust you?” explaining the predictions of any classifier. In: Proceedings of 22nd ACM SIGKDD, pp. 1135–1144 (2016)
Riedelbauch, D.: Dynamic Task Sharing for Flexible Human-Robot Teaming under Partial Workspace Observability. Ph.D. thesis, University of Bayreuth (2020)
Riedelbauch, D., Henrich, D.: Fast graphical task modelling for flexible human-robot teaming. In: 50th International Symposium on Robotics (ISR), pp. 1–6. VDE (2018)
Riedelbauch, D., Hümmer, J.: A benchmark toolkit for collaborative human-robot interaction. In: 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 806–813. IEEE (2022)
Riedelbauch, D., Sucker., S.: Visual programming of robot tasks with product and process variety. In: Annals of Scientific Society for Assembly, Handling and Industrial Robotics (to appear) (2022)
Scheer, A.W., Thomas, O., Adam, O.: Process modeling using event-driven process chains. In: Process-Aware Information Systems: Bridging People and Software through Process Technology, pp. 119–145 (2005)
Selic, B., et al.: Omg unified modeling language (version 2.5). Technical Report (2015)
Senft, E., et al.: Situated live programming for human-robot collaboration. In: ACM Symposium on User Interface Software and Technology, pp. 613–625 (2021)
Steinmetz, F., Wollschläger, A., Weitschat, R.: Razer - a HRI for visual task-level programming and intuitive skill parameterization. IEEE Robot. Autom. Lett. 3(3), 1362–1369 (2018)
Thomas, U., et al.: A new skill based robot programming language using UML/P statecharts. In: IEEE International Conference on Robotics and Automation (2013)
Van Der Aalst, W.M., Ter Hofstede, A.H., Weske, M.: Business process management: a survey. Bus. Process Manage. 2678(1019), 1–12 (2003)
Wiedmann, P.C.K.: Agiles Geschäftsprozessmanagement auf Basis gebrauchssprachlicher Modellierung. Universitaet Bayreuth (Germany) (2017)
Acknowledgements
We thank Philipp Jahn and Carsten Scholle for their valuable work supporting the implementation and evaluation of our approach.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Fichtner, M., Sucker, S., Riedelbauch, D., Jablonski, S., Henrich, D. (2024). Enriching Process Models with Relevant Process Details for Flexible Human-Robot Teaming. In: Gao, H., Wang, X., Voros, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 563. Springer, Cham. https://doi.org/10.1007/978-3-031-54531-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-54531-3_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54530-6
Online ISBN: 978-3-031-54531-3
eBook Packages: Computer ScienceComputer Science (R0)