Abstract
In the process engineering reliable and trustworthy AI systems there is significant wisdom to be gained from traditional engineering domains. Extending on earlier work our attention is on topics that stress the principles of building human-AI systems. We plea for a reinforced attention for engineering methods and processes in order to urge the essence for improved scientific progress and industrial AI applications where one can stand on the shoulders of giants. On the one hand, we see their complexity increase on an individual level, as well as on their connected dependency levels, whilst on the other hand, we see a growing lack of experience on the level of their design and engineering. The complexity of current AI models often limits our understanding. The methods and processes to ensure safety, reliability, and transparency are insufficient. This poses serious risks at the level of trustworthiness, particularly when it comes to critical applications with significant social, economic or even physical impact. Future AI systems must adhere to stringent requirements, as mandated, for instance, by the European AI Act, ensuring meticulous design, validation, and certification based on clearly defined criteria.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
“Model” is an ambiguous term commonly used in the field of machine learning and data science. We distinguish statistical models (mainly used in data driven engineering approaches) from semantic models, commonly used in the field of knowledge engineering. Here, we refer to the latter. See also in [70] for a unified taxonomy of AI.
- 2.
Also here an additional point regarding the term ambiguous term ’model’: we employ the word model here to signify an engineering framework, often referred to as a ’maturity model’.
References
Akata, Z., Balliet, D., de Rijke, M., Dignum, F., Dignum, V., Eiben, G., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., Hung, H., Jonker, C., Monz, C., Neerincx, M., Oliehoek, F., Prakken, H., Schlobach, S., van der Gaag, L., van Harmelen, F., van Hoof, H., van Riemsdijk, B., van Wynsberghe, A., Verbrugge, R., Verheij, B., Vossen, P., Welling, M.: A research agenda for hybrid intelligence: augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer 53(08), 18–28 (2020). IEEE Computer Society
Angelidou, M., Politis, C., Panori, A., Bakratsas, T., Fellnhofer, K.: Emerging smart city, transport and energy trends in urban settings: results of a pan-European foresight exercise with 120 experts. Technol. Forecast. Soc. Chang. 183, 121915 (2022). October
Antoniou, G., Van Harmelen, F.: A Semantic Web Primer. MIT Press (2004)
Bader, S., Hitzler, P.: Dimensions of neural-symbolic integration - a structured survey (2005). arXiv:cs/0511042 version: 1
Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind’’ ? Cognition 21(1), 37–46 (1985). October
Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M., Krishna, R., Kuditipudi, R., Kumar, A., Ladhak, F., Lee, M., Lee, T., Leskovec, J., Levent, I., Li, X.L., Li, X., Ma, T., Malik, A., Manning, C.D., Mirchandani, S., Mitchell, E., Munyikwa, Z., Nair, S., Narayan, A., Narayanan, D., Newman, B., Nie, A., Niebles, J.C., Nilforoshan, H., Nyarko, J., Ogut, G., Orr, L., Papadimitriou, I., Park, J.S., Piech, C., Portelance, E., Potts, C., Raghunathan, A., Reich, R., Ren, H., Rong, F., Roohani, Y., Ruiz, C., Ryan, J., Ré, C., Sadigh, D., Sagawa, S., Santhanam, K., Shih, A., Srinivasan, K., Tamkin, A., Taori, R., Thomas, A.W., Tramèr, F., Wang, R.E., Wang, W., Wu, B., Wu, J., Wu, Y., Xie, S.M., Yasunaga, M., You, J., Zaharia, M., Zhang, M., Zhang, T., Zhang, X., Zhang, Y., Zheng, L., Zhou, K., Liang, P.: On the opportunities and risks of foundation models, July 2022. arXiv:2108.07258 [cs]
Buehler, M.C., Weisswange, T.H.: Theory of mind based communication for human agent cooperation. In: 2020 IEEE International Conference on Human-Machine Systems (ICHMS), pp. 1–6 (2020)
Byom, L., Mutlu, B.: Theory of mind: mechanisms, methods, and new directions. Front. Hum. Neurosci. 7 (2013)
Carloni, G., Berti, A., Colantonio, S.: The role of causality in explainable artificial intelligence (2023). arXiv:2309.09901 [cs]
Chan, S., Siegel, E.L.: Will machine learning end the viability of radiology as a thriving medical specialty? Br. J. Radiol. 92(1094), 20180416 (2019). February
Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends® Signal Process. 7(3–4), 197–387 (2014)
Dennett, D.C.: The Intentional Stance, p. xi, 388. The MIT Press, Cambridge, MA, US (1987)
Dunin-Keplicz, B.M., Verbrugge, R.: Teamwork in Multi-Agent Systems: A Formal Approach, 1st edn. Wiley Publishing (2010)
Eberhardt, F.: Introduction to the foundations of causal discovery. Int. J. Data Sci. Anal. 3(2), 81–91 (2017). March
Emelin, D., Le Bras, R., Hwang, J.D., Forbes, M., Choi, Y., Moral stories: situated reasoning about norms, intents, actions, and their consequences. arXiv:2012.15738 [cs], arXiv: 2012.15738 (2020)
Fensel, D., van Harmelen, F., Horrocks, I., McGuinness, D.L., Patel-Schneider, P.F.: OIL: an ontology infrastructure for the Semantic Web. IEEE Intell. Syst. 16(2), 38–45 (2001). Conference Name: IEEE Intelligent Systems
Frith, C., Frith, U.: Theory of mind. Curr. Biol. 15(17), R644–R645 (2005)
Gamma, E., Helm, R., Johnson, R., Vlissides, J., Booch, G.: Design Patterns: Elements of Reusable Object-Oriented Software, 1st edn. Addison-Wesley Professional, Reading, MA (1994)
Ganesh, S., Beucler, T., Tam, F.I.-H., Gomez, M.S., Runge, J., Gerhardus, A.: Selecting robust features for machine-learning applications using multidata causal discovery. Environ. Data Sci. 2, e27 (2023)
d’Avila Garcez, A., Broda, K.B., Gabbay, D.M.: Neural-symbolic integration: the road ahead. In: d’Avila Garcez, A., Broda, K.B., Gabbay, D.M. (eds.) Neural-Symbolic Learning Systems: Foundations and Applications, Perspectives in Neural Computing, pp. 235–252. Springer, London (2002)
d’Avila Garcez, A., Broda, K.B., Gabbay, D.M.: Neural-symbolic integration: the road ahead. In: d’Avila Garcez, A., Broda, K.B., Gabbay, D.M. (eds.) Neural-Symbolic Learning Systems: Foundations and Applications, Perspectives in Neural Computing, pp. 235–252. Springer, London (2002)
d’Avila Garcez, A., Lamb, L.C.: Neurosymbolic AI: the 3rd wave. Artif. Intell. Rev. 56(11):12387–12406 (2023)
d’Avila Garcez, A.S., Gabbay, D.M., Broda, K.B.: Neural-Symbolic Learning System: Foundations and Applications. Springer-Verlag, Berlin, Heidelberg (2002)
Gärdenfors, P.: From sensations to concepts: a proposal for two learning processes. Rev. Philos. Psychol. 10(3), 441–464 (2019). September
Harbers, M., Verbrugge, R., Sierra, C., Debenham, J.: The examination of an information-based approach to trust. In: Sichman, J.S., Padget, J., Ossowski, S., Noriega, P. (eds.) Coordination, Organizations, Institutions, and Norms in Agent Systems III, Lecture Notes in Computer Science, pp. 71–82. Springer, Berlin, Heidelberg (2008)
Harnad, S.: The symbol grounding problem. Physica D 42(1), 335–346 (1990). June
Hashem, I.A.T., Usmani, R.S.A., Almutairi, M.S., Ibrahim, A.O., Zakari, A., Alotaibi, F., Alhashmi, S.M., Chiroma, H.: Urban computing for sustainable smart cities: recent advances, taxonomy, and open research challenges. Sustainability 15(5), 3916 (2023). Number: 5 Publisher: Multidisciplinary Digital Publishing Institute
Hasthanasombat, A.: A causal perspective on model robustness: case studies in health and sensor data. Ph.D. thesis, University of Cambridge, Cambridge, UK (2022)
Haynes, C., Luck, M., McBurney, P., Mahmoud, S., Vítek, T., Miles, S.: Engineering the emergence of norms: a review. Knowl. Eng. Rev. 32 (2017)
Ismael, J.: Reflections on the asymmetry of causation. Interface Focus textbf13(3), 20220081 (2023)
Janssen, S., Sharpanskykh, A., Sahand Mohammadi Ziabari, S.: Using causal discovery to design agent-based models. In: Van Dam, K.H., Verstaevel, N. (eds.) Multi-agent-Based Simulation XXII, Lecture Notes in Computer Science, pp. 15–28. Springer International Publishing, Cham (2022)
Jiang, L., Hwang, J.D., Bhagavatula, C., Le Bras, R., Forbes, M., Borchardt, J., Liang, J., Etzioni, O., Sap, M., Choi, Y.: Delphi: towards machine ethics and norms. arXiv:2110.07574 [cs]. arXiv: 2110.07574 (2021)
Kahneman, D.: Thinking. Fast and Slow. Farrar. Straus and Giroux, New York (2011). October
Kutach, D.: Causal asymmetry. In: Kutach, D. (ed) Causation and its Basis in Fundamental Physics, p 0. Oxford University Press (2013)
Kyono, T., van der Schaar, M.: Improving model robustness using causal knowledge (2019). arXiv:1911.12441 [cs, stat]
Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). Number: 7553 Publisher: Nature Publishing Group
Lévi-Strauss, C.: La pensée sauvage. Plon (1962). Google-Books-ID: OoEeAAAAIAAJ
Marcus, G.: The next decade in AI: four steps towards robust artificial intelligence. arXiv:2002.06177 [cs]. arXiv: 2002.06177 (2020)
Marcus, G.: Deep Learning Is Hitting a Wall (2022)
Marcus, G., Davis, E.: Rebooting AI: Building Artificial Intelligence We Can Trust. Vintage (2019)
Mazaheri, B., Mastakouri, A., Janzing, D., Hardt, M.: Causal information splitting: engineering proxy features for robustness to distribution shifts. In: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, pp. 1401–1411. PMLR (2023). ISSN: 2640-3498
Meyer-Vitali, A.: AI Engineering for Trust by Design. In: Proceedings of the 12th International Conference on Model-Based Software and Systems Engineering (MODELSWARD 2024), pp. 357–364, Rome, Italy. SCITEPRESS—Science and Technology Publications, Lda (2024)
Meyer-Vitali, A., Mulder, W., de Boer, M.H.T.: Modular design patterns for hybrid actors. In: Cooperative AI Workshop, volume 2021 of NeurIPS (2021). arXiv: 2109.09331
Morocho-Cayamcela, M.E., Lee, H., Lim, W.: Machine learning for 5G/B5G mobile and wireless communications: potential, limitations, and future directions. IEEE Access 7, 137184–137206 (2019)
Mulder, W., Meyer-Vitali, A.: A maturity model for collaborative agents in human-AI ecosystems. In: Camarinha-Matos, L.M., Boucher, X., Ortiz, A., (eds.) Collaborative Networks in Digitalization and Society 5.0, IFIP Advances in Information and Communication Technology, pp. 328–335. Springer Nature Switzerland, Cham (2023)
Oliveira, G.M., Vidal, D.G., Ferraz, M.P.: Urban lifestyles and consumption patterns. In: Filho, W.L., Azul, A.M., Brandli, L., Özuyar, P.G., Wall, T. (eds.) Sustainable Cities and Communities, Encyclopedia of the UN Sustainable Development Goals, pp. 851–860. Springer International Publishing, Cham (2020)
Pearl, J.: An introduction to causal inference. Int. J. Biostat. 6(2) (2010)
Pearl, J.: The seven tools of causal inference, with reflections on machine learning. Commun. ACM 62(3), 54–60 (2019). February
Pearl, J.: Radical empiricism and machine learning research. J. Causal Infer. 9(1), 78–82 (2021)
Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. John Wiley & Sons (2016). Google-Books-ID: I0V2CwAAQBAJ
Pearl, J., Mackenzie, D.: The Book of Why: The New Science of Cause and Effect, 1st edn. Basic Books, New York (2018)
Petrikovičová, L., Kurilenko, V., Akimjak, A., Akimjaková, B., Majda, P., Ďatelinka, A., Biryukova, Y., Hlad, L., Kondrla, P., Maryanovich, D., Ippolitova, L., Roubalová, M., Petrikovič, J.: Is the size of the city important for the quality of urban life? Comparison of a small and a large city. Sustainability 14(23), 15589 (2022). Number: 23 Publisher: Multidisciplinary Digital Publishing Institute
Popelka, S.: Laura Narvaez Zertuche, and Hubert Beroche. Urban AI Guide. Technical report, Zenodo (2023)
Premack, D., Woodruff, G.: Does the chimpanzee have a theory of mind? Behav. Brain Sci. 1(4), 515–526. Publisher, Cambridge University Press (1978)
Price, H.: Agency and causal asymmetry. Mind 101(403), 501–520 (1992). Publisher: [Oxford University Press, Mind Association]
Ramchurn, S.D., Stein, S., Jennings, N.R.: Trustworthy human-AI partnerships. iScience 24(8), 102891 (2021). August
Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: Proceedings of the First International Conference on Multiagent Systems, 1995
Rawal, A., Raglin, A., Sadler, B.M., Rawat, D.B.: Explainability and causality for robust, fair, and trustworthy artificial reasoning. In: Artificial Intelligence and Machine Learning for Multi-domain Operations Applications V, vol. 12538, pp. 493–500. SPIE (2023)
Ross, L.N., Bassett, D.S.: Causation in neuroscience: keeping mechanism meaningful. Nat. Rev. Neurosci., pp. 1–10 (2024)
Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. arXiv:1811.10154 [cs, stat]. arXiv: 1811.10154 (2019)
Sadiq, R.B., Safie, N., Rahman, A.H.A., Goudarzi, S.: Artificial intelligence maturity model: a systematic literature review. PeerJ Comput. Sci. 7, e661 (2021)
Savarimuthu, B.T.R., Cranefield, S., Purvis, M., Purvis, M.: Role model based mechanism for norm emergence in artificial agent societies. In: Sichman, J.S., Padget, J., Ossowski, S., Noriega, P. (eds.) Coordination, Organizations, Institutions, and Norms in Agent Systems III, Lecture Notes in Computer Science, pp. 203–217. Springer, Berlin, Heidelberg (2008)
Schölkopf, B., Locatello, F., Bauer, S., Ke, N.R., Kalchbrenner, N., Goyal, A., Bengio, Y.: Toward causal representation learning. Proc. IEEE 109(5), 612–634 (2021). Conference Name: Proceedings of the IEEE
Schölkopf, B., von Kügelgen, J.: From statistical to causal learning (2022). arXiv:2204.00607 [cs, stat]
Searle, J.R.: Minds, brains, and programs. Behav. Brain Sci. 3(3), 417–424 (1980)
Shen, X., Bühlmann, P., Taeb, A.: Causality-oriented robustness: exploiting general additive interventions (2023). arXiv:2307.10299 [cs, stat]
Thiebes, S., Lins, S., Sunyaev, A.: Trustworthy artificial intelligence. Electron. Mark. 31(2), 447–464 (2021)
Tiddi, I., De Boer, V., Schlobach, S., Meyer-Vitali, A.: Knowledge engineering for hybrid intelligence. In: Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP’23, pp. 75–82. Association for Computing Machinery, New York, NY, USA (2023)
van Bekkum, M., de Boer, M., van Harmelen, F., Meyer-Vitali, A., ten Teije, A.: Modular design patterns for hybrid learning and reasoning systems. Appl. Intell. 51(9), 6528–6546 (2021)
van Stijn, J.J., Neerincx, M.A., ten Teije, A., Vethman, S.: Team design patterns for moral decisions in hybrid intelligent systems: 2021 AAAI Spring Symposium on Combining Machine Learning and Knowledge Engineering, AAAI-MAKE 2021. In: AAAI-MAKE 2021 Combining Machine Learning and Knowledge Engineering, pp. 1–12. CEUR-WS (2021)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N. Kaiser, L., Polosukhin, I.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need (2023). arXiv:1706.03762 [cs]
Verbrugge, R.: Testing and training theory of mind for hybrid human-agent environments. In: Rocha, A.P., Steels, L., van den Herik, H.J. (eds.) Proceedings of the 12th International Conference on Agents and Artificial Intelligence, ICAART 2020, Volume 1, Valletta, Malta, February 22–24, 2020, p. 11. SCITEPRESS, 2020
Verbrugge, R., Mol, L.: Learning to apply theory of mind. J. Logic Lang. Inform. 17(4), 489–511 (2008)
Verkijk, S., Roothaert, R., Pernisch, R., Schlobach, S.: Do you catch my drift? On the usage of embedding methods to measure concept shift in knowledge graphs. In: Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP’23, pp. 70–74. Association for Computing Machinery, New York, NY, USA (2023)
Vlontzos, A., Kainz, B., Gilligan-Lee, C.M.: Estimating categorical counterfactuals via deep twin networks. Nat. Mach. Intell. 5(2), 159–168 (2023)
WBGU—German Advisory Council on Global Change. Humanity on the move: Unlocking the transformative power of cities. Technical report, WBGU, Berlin, 2016. Frauke Kraas, Claus Leggewie, Peter Lemke, Ellen Matthies, Dirk Messner, Nebojsa Nakicenovic, Hans Joachim Schellnhuber, Sabine Schlacke, Uwe Schneidewind
Yang, S., Yu, K., Cao, F., Liu, L., Wang, H., Li, J.: Learning causal representations for robust domain adaptation. IEEE Trans. Knowl. Data Eng. 35(3), 2750–2764, (2023). Conference Name: IEEE Transactions on Knowledge and Data Engineering
Yu, D., Yang, B., Liu, D., Wang, H., Pan, S.: A survey on neural-symbolic learning systems (2021)
Zhang, C., Zhang, K., Li, Y.: A causal view on robustness of neural networks. Adv. Neural Inf. Process. Syst. 33, 289–301 (2020)
Zhu, S., Ng, I., Chen, Z.: Causal discovery with reinforcement learning. In: International Conference on Learning Representations, Online, April 2020
Acknowledgments
This research was partially supported by TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Meyer-Vitali, A., Mulder, W. (2024). Engineering Principles for Building Trusted Human-AI Systems. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2024. Lecture Notes in Networks and Systems, vol 1066. Springer, Cham. https://doi.org/10.1007/978-3-031-66428-1_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-66428-1_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-66427-4
Online ISBN: 978-3-031-66428-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)