Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Engineering Principles for Building Trusted Human-AI Systems

  • Conference paper
  • First Online:
Intelligent Systems and Applications (IntelliSys 2024)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 1066))

Included in the following conference series:

  • 291 Accesses

Abstract

In the process engineering reliable and trustworthy AI systems there is significant wisdom to be gained from traditional engineering domains. Extending on earlier work our attention is on topics that stress the principles of building human-AI systems. We plea for a reinforced attention for engineering methods and processes in order to urge the essence for improved scientific progress and industrial AI applications where one can stand on the shoulders of giants. On the one hand, we see their complexity increase on an individual level, as well as on their connected dependency levels, whilst on the other hand, we see a growing lack of experience on the level of their design and engineering. The complexity of current AI models often limits our understanding. The methods and processes to ensure safety, reliability, and transparency are insufficient. This poses serious risks at the level of trustworthiness, particularly when it comes to critical applications with significant social, economic or even physical impact. Future AI systems must adhere to stringent requirements, as mandated, for instance, by the European AI Act, ensuring meticulous design, validation, and certification based on clearly defined criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 159.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    “Model” is an ambiguous term commonly used in the field of machine learning and data science. We distinguish statistical models (mainly used in data driven engineering approaches) from semantic models, commonly used in the field of knowledge engineering. Here, we refer to the latter. See also in [70] for a unified taxonomy of AI.

  2. 2.

    Also here an additional point regarding the term ambiguous term ’model’: we employ the word model here to signify an engineering framework, often referred to as a ’maturity model’.

References

  1. Akata, Z., Balliet, D., de Rijke, M., Dignum, F., Dignum, V., Eiben, G., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., Hung, H., Jonker, C., Monz, C., Neerincx, M., Oliehoek, F., Prakken, H., Schlobach, S., van der Gaag, L., van Harmelen, F., van Hoof, H., van Riemsdijk, B., van Wynsberghe, A., Verbrugge, R., Verheij, B., Vossen, P., Welling, M.: A research agenda for hybrid intelligence: augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer 53(08), 18–28 (2020). IEEE Computer Society

    Google Scholar 

  2. Angelidou, M., Politis, C., Panori, A., Bakratsas, T., Fellnhofer, K.: Emerging smart city, transport and energy trends in urban settings: results of a pan-European foresight exercise with 120 experts. Technol. Forecast. Soc. Chang. 183, 121915 (2022). October

    Article  Google Scholar 

  3. Antoniou, G., Van Harmelen, F.: A Semantic Web Primer. MIT Press (2004)

    Google Scholar 

  4. Bader, S., Hitzler, P.: Dimensions of neural-symbolic integration - a structured survey (2005). arXiv:cs/0511042 version: 1

  5. Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind’’ ? Cognition 21(1), 37–46 (1985). October

    Article  Google Scholar 

  6. Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J.Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D.E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P.W., Krass, M., Krishna, R., Kuditipudi, R., Kumar, A., Ladhak, F., Lee, M., Lee, T., Leskovec, J., Levent, I., Li, X.L., Li, X., Ma, T., Malik, A., Manning, C.D., Mirchandani, S., Mitchell, E., Munyikwa, Z., Nair, S., Narayan, A., Narayanan, D., Newman, B., Nie, A., Niebles, J.C., Nilforoshan, H., Nyarko, J., Ogut, G., Orr, L., Papadimitriou, I., Park, J.S., Piech, C., Portelance, E., Potts, C., Raghunathan, A., Reich, R., Ren, H., Rong, F., Roohani, Y., Ruiz, C., Ryan, J., Ré, C., Sadigh, D., Sagawa, S., Santhanam, K., Shih, A., Srinivasan, K., Tamkin, A., Taori, R., Thomas, A.W., Tramèr, F., Wang, R.E., Wang, W., Wu, B., Wu, J., Wu, Y., Xie, S.M., Yasunaga, M., You, J., Zaharia, M., Zhang, M., Zhang, T., Zhang, X., Zhang, Y., Zheng, L., Zhou, K., Liang, P.: On the opportunities and risks of foundation models, July 2022. arXiv:2108.07258 [cs]

  7. Buehler, M.C., Weisswange, T.H.: Theory of mind based communication for human agent cooperation. In: 2020 IEEE International Conference on Human-Machine Systems (ICHMS), pp. 1–6 (2020)

    Google Scholar 

  8. Byom, L., Mutlu, B.: Theory of mind: mechanisms, methods, and new directions. Front. Hum. Neurosci. 7 (2013)

    Google Scholar 

  9. Carloni, G., Berti, A., Colantonio, S.: The role of causality in explainable artificial intelligence (2023). arXiv:2309.09901 [cs]

  10. Chan, S., Siegel, E.L.: Will machine learning end the viability of radiology as a thriving medical specialty? Br. J. Radiol. 92(1094), 20180416 (2019). February

    Article  Google Scholar 

  11. Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends® Signal Process. 7(3–4), 197–387 (2014)

    Google Scholar 

  12. Dennett, D.C.: The Intentional Stance, p. xi, 388. The MIT Press, Cambridge, MA, US (1987)

    Google Scholar 

  13. Dunin-Keplicz, B.M., Verbrugge, R.: Teamwork in Multi-Agent Systems: A Formal Approach, 1st edn. Wiley Publishing (2010)

    Google Scholar 

  14. Eberhardt, F.: Introduction to the foundations of causal discovery. Int. J. Data Sci. Anal. 3(2), 81–91 (2017). March

    Article  Google Scholar 

  15. Emelin, D., Le Bras, R., Hwang, J.D., Forbes, M., Choi, Y., Moral stories: situated reasoning about norms, intents, actions, and their consequences. arXiv:2012.15738 [cs], arXiv: 2012.15738 (2020)

  16. Fensel, D., van Harmelen, F., Horrocks, I., McGuinness, D.L., Patel-Schneider, P.F.: OIL: an ontology infrastructure for the Semantic Web. IEEE Intell. Syst. 16(2), 38–45 (2001). Conference Name: IEEE Intelligent Systems

    Google Scholar 

  17. Frith, C., Frith, U.: Theory of mind. Curr. Biol. 15(17), R644–R645 (2005)

    Google Scholar 

  18. Gamma, E., Helm, R., Johnson, R., Vlissides, J., Booch, G.: Design Patterns: Elements of Reusable Object-Oriented Software, 1st edn. Addison-Wesley Professional, Reading, MA (1994)

    Google Scholar 

  19. Ganesh, S., Beucler, T., Tam, F.I.-H., Gomez, M.S., Runge, J., Gerhardus, A.: Selecting robust features for machine-learning applications using multidata causal discovery. Environ. Data Sci. 2, e27 (2023)

    Google Scholar 

  20. d’Avila Garcez, A., Broda, K.B., Gabbay, D.M.: Neural-symbolic integration: the road ahead. In: d’Avila Garcez, A., Broda, K.B., Gabbay, D.M. (eds.) Neural-Symbolic Learning Systems: Foundations and Applications, Perspectives in Neural Computing, pp. 235–252. Springer, London (2002)

    Google Scholar 

  21. d’Avila Garcez, A., Broda, K.B., Gabbay, D.M.: Neural-symbolic integration: the road ahead. In: d’Avila Garcez, A., Broda, K.B., Gabbay, D.M. (eds.) Neural-Symbolic Learning Systems: Foundations and Applications, Perspectives in Neural Computing, pp. 235–252. Springer, London (2002)

    Google Scholar 

  22. d’Avila Garcez, A., Lamb, L.C.: Neurosymbolic AI: the 3rd wave. Artif. Intell. Rev. 56(11):12387–12406 (2023)

    Google Scholar 

  23. d’Avila Garcez, A.S., Gabbay, D.M., Broda, K.B.: Neural-Symbolic Learning System: Foundations and Applications. Springer-Verlag, Berlin, Heidelberg (2002)

    Google Scholar 

  24. Gärdenfors, P.: From sensations to concepts: a proposal for two learning processes. Rev. Philos. Psychol. 10(3), 441–464 (2019). September

    Article  Google Scholar 

  25. Harbers, M., Verbrugge, R., Sierra, C., Debenham, J.: The examination of an information-based approach to trust. In: Sichman, J.S., Padget, J., Ossowski, S., Noriega, P. (eds.) Coordination, Organizations, Institutions, and Norms in Agent Systems III, Lecture Notes in Computer Science, pp. 71–82. Springer, Berlin, Heidelberg (2008)

    Google Scholar 

  26. Harnad, S.: The symbol grounding problem. Physica D 42(1), 335–346 (1990). June

    Article  Google Scholar 

  27. Hashem, I.A.T., Usmani, R.S.A., Almutairi, M.S., Ibrahim, A.O., Zakari, A., Alotaibi, F., Alhashmi, S.M., Chiroma, H.: Urban computing for sustainable smart cities: recent advances, taxonomy, and open research challenges. Sustainability 15(5), 3916 (2023). Number: 5 Publisher: Multidisciplinary Digital Publishing Institute

    Google Scholar 

  28. Hasthanasombat, A.: A causal perspective on model robustness: case studies in health and sensor data. Ph.D. thesis, University of Cambridge, Cambridge, UK (2022)

    Google Scholar 

  29. Haynes, C., Luck, M., McBurney, P., Mahmoud, S., Vítek, T., Miles, S.: Engineering the emergence of norms: a review. Knowl. Eng. Rev. 32 (2017)

    Google Scholar 

  30. Ismael, J.: Reflections on the asymmetry of causation. Interface Focus textbf13(3), 20220081 (2023)

    Google Scholar 

  31. Janssen, S., Sharpanskykh, A., Sahand Mohammadi Ziabari, S.: Using causal discovery to design agent-based models. In: Van Dam, K.H., Verstaevel, N. (eds.) Multi-agent-Based Simulation XXII, Lecture Notes in Computer Science, pp. 15–28. Springer International Publishing, Cham (2022)

    Google Scholar 

  32. Jiang, L., Hwang, J.D., Bhagavatula, C., Le Bras, R., Forbes, M., Borchardt, J., Liang, J., Etzioni, O., Sap, M., Choi, Y.: Delphi: towards machine ethics and norms. arXiv:2110.07574 [cs]. arXiv: 2110.07574 (2021)

  33. Kahneman, D.: Thinking. Fast and Slow. Farrar. Straus and Giroux, New York (2011). October

    Google Scholar 

  34. Kutach, D.: Causal asymmetry. In: Kutach, D. (ed) Causation and its Basis in Fundamental Physics, p 0. Oxford University Press (2013)

    Google Scholar 

  35. Kyono, T., van der Schaar, M.: Improving model robustness using causal knowledge (2019). arXiv:1911.12441 [cs, stat]

  36. Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017)

    Google Scholar 

  37. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). Number: 7553 Publisher: Nature Publishing Group

    Google Scholar 

  38. Lévi-Strauss, C.: La pensée sauvage. Plon (1962). Google-Books-ID: OoEeAAAAIAAJ

    Google Scholar 

  39. Marcus, G.: The next decade in AI: four steps towards robust artificial intelligence. arXiv:2002.06177 [cs]. arXiv: 2002.06177 (2020)

  40. Marcus, G.: Deep Learning Is Hitting a Wall (2022)

    Google Scholar 

  41. Marcus, G., Davis, E.: Rebooting AI: Building Artificial Intelligence We Can Trust. Vintage (2019)

    Google Scholar 

  42. Mazaheri, B., Mastakouri, A., Janzing, D., Hardt, M.: Causal information splitting: engineering proxy features for robustness to distribution shifts. In: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, pp. 1401–1411. PMLR (2023). ISSN: 2640-3498

    Google Scholar 

  43. Meyer-Vitali, A.: AI Engineering for Trust by Design. In: Proceedings of the 12th International Conference on Model-Based Software and Systems Engineering (MODELSWARD 2024), pp. 357–364, Rome, Italy. SCITEPRESS—Science and Technology Publications, Lda (2024)

    Google Scholar 

  44. Meyer-Vitali, A., Mulder, W., de Boer, M.H.T.: Modular design patterns for hybrid actors. In: Cooperative AI Workshop, volume 2021 of NeurIPS (2021). arXiv: 2109.09331

  45. Morocho-Cayamcela, M.E., Lee, H., Lim, W.: Machine learning for 5G/B5G mobile and wireless communications: potential, limitations, and future directions. IEEE Access 7, 137184–137206 (2019)

    Google Scholar 

  46. Mulder, W., Meyer-Vitali, A.: A maturity model for collaborative agents in human-AI ecosystems. In: Camarinha-Matos, L.M., Boucher, X., Ortiz, A., (eds.) Collaborative Networks in Digitalization and Society 5.0, IFIP Advances in Information and Communication Technology, pp. 328–335. Springer Nature Switzerland, Cham (2023)

    Google Scholar 

  47. Oliveira, G.M., Vidal, D.G., Ferraz, M.P.: Urban lifestyles and consumption patterns. In: Filho, W.L., Azul, A.M., Brandli, L., Özuyar, P.G., Wall, T. (eds.) Sustainable Cities and Communities, Encyclopedia of the UN Sustainable Development Goals, pp. 851–860. Springer International Publishing, Cham (2020)

    Google Scholar 

  48. Pearl, J.: An introduction to causal inference. Int. J. Biostat. 6(2) (2010)

    Google Scholar 

  49. Pearl, J.: The seven tools of causal inference, with reflections on machine learning. Commun. ACM 62(3), 54–60 (2019). February

    Article  Google Scholar 

  50. Pearl, J.: Radical empiricism and machine learning research. J. Causal Infer. 9(1), 78–82 (2021)

    Google Scholar 

  51. Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. John Wiley & Sons (2016). Google-Books-ID: I0V2CwAAQBAJ

    Google Scholar 

  52. Pearl, J., Mackenzie, D.: The Book of Why: The New Science of Cause and Effect, 1st edn. Basic Books, New York (2018)

    Google Scholar 

  53. Petrikovičová, L., Kurilenko, V., Akimjak, A., Akimjaková, B., Majda, P., Ďatelinka, A., Biryukova, Y., Hlad, L., Kondrla, P., Maryanovich, D., Ippolitova, L., Roubalová, M., Petrikovič, J.: Is the size of the city important for the quality of urban life? Comparison of a small and a large city. Sustainability 14(23), 15589 (2022). Number: 23 Publisher: Multidisciplinary Digital Publishing Institute

    Google Scholar 

  54. Popelka, S.: Laura Narvaez Zertuche, and Hubert Beroche. Urban AI Guide. Technical report, Zenodo (2023)

    Google Scholar 

  55. Premack, D., Woodruff, G.: Does the chimpanzee have a theory of mind? Behav. Brain Sci. 1(4), 515–526. Publisher, Cambridge University Press (1978)

    Google Scholar 

  56. Price, H.: Agency and causal asymmetry. Mind 101(403), 501–520 (1992). Publisher: [Oxford University Press, Mind Association]

    Google Scholar 

  57. Ramchurn, S.D., Stein, S., Jennings, N.R.: Trustworthy human-AI partnerships. iScience 24(8), 102891 (2021). August

    Article  Google Scholar 

  58. Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: Proceedings of the First International Conference on Multiagent Systems, 1995

    Google Scholar 

  59. Rawal, A., Raglin, A., Sadler, B.M., Rawat, D.B.: Explainability and causality for robust, fair, and trustworthy artificial reasoning. In: Artificial Intelligence and Machine Learning for Multi-domain Operations Applications V, vol. 12538, pp. 493–500. SPIE (2023)

    Google Scholar 

  60. Ross, L.N., Bassett, D.S.: Causation in neuroscience: keeping mechanism meaningful. Nat. Rev. Neurosci., pp. 1–10 (2024)

    Google Scholar 

  61. Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. arXiv:1811.10154 [cs, stat]. arXiv: 1811.10154 (2019)

  62. Sadiq, R.B., Safie, N., Rahman, A.H.A., Goudarzi, S.: Artificial intelligence maturity model: a systematic literature review. PeerJ Comput. Sci. 7, e661 (2021)

    Google Scholar 

  63. Savarimuthu, B.T.R., Cranefield, S., Purvis, M., Purvis, M.: Role model based mechanism for norm emergence in artificial agent societies. In: Sichman, J.S., Padget, J., Ossowski, S., Noriega, P. (eds.) Coordination, Organizations, Institutions, and Norms in Agent Systems III, Lecture Notes in Computer Science, pp. 203–217. Springer, Berlin, Heidelberg (2008)

    Google Scholar 

  64. Schölkopf, B., Locatello, F., Bauer, S., Ke, N.R., Kalchbrenner, N., Goyal, A., Bengio, Y.: Toward causal representation learning. Proc. IEEE 109(5), 612–634 (2021). Conference Name: Proceedings of the IEEE

    Google Scholar 

  65. Schölkopf, B., von Kügelgen, J.: From statistical to causal learning (2022). arXiv:2204.00607 [cs, stat]

  66. Searle, J.R.: Minds, brains, and programs. Behav. Brain Sci. 3(3), 417–424 (1980)

    Google Scholar 

  67. Shen, X., Bühlmann, P., Taeb, A.: Causality-oriented robustness: exploiting general additive interventions (2023). arXiv:2307.10299 [cs, stat]

  68. Thiebes, S., Lins, S., Sunyaev, A.: Trustworthy artificial intelligence. Electron. Mark. 31(2), 447–464 (2021)

    Article  Google Scholar 

  69. Tiddi, I., De Boer, V., Schlobach, S., Meyer-Vitali, A.: Knowledge engineering for hybrid intelligence. In: Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP’23, pp. 75–82. Association for Computing Machinery, New York, NY, USA (2023)

    Google Scholar 

  70. van Bekkum, M., de Boer, M., van Harmelen, F., Meyer-Vitali, A., ten Teije, A.: Modular design patterns for hybrid learning and reasoning systems. Appl. Intell. 51(9), 6528–6546 (2021)

    Article  Google Scholar 

  71. van Stijn, J.J., Neerincx, M.A., ten Teije, A., Vethman, S.: Team design patterns for moral decisions in hybrid intelligent systems: 2021 AAAI Spring Symposium on Combining Machine Learning and Knowledge Engineering, AAAI-MAKE 2021. In: AAAI-MAKE 2021 Combining Machine Learning and Knowledge Engineering, pp. 1–12. CEUR-WS (2021)

    Google Scholar 

  72. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N. Kaiser, L., Polosukhin, I.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  73. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need (2023). arXiv:1706.03762 [cs]

  74. Verbrugge, R.: Testing and training theory of mind for hybrid human-agent environments. In: Rocha, A.P., Steels, L., van den Herik, H.J. (eds.) Proceedings of the 12th International Conference on Agents and Artificial Intelligence, ICAART 2020, Volume 1, Valletta, Malta, February 22–24, 2020, p. 11. SCITEPRESS, 2020

    Google Scholar 

  75. Verbrugge, R., Mol, L.: Learning to apply theory of mind. J. Logic Lang. Inform. 17(4), 489–511 (2008)

    Article  MathSciNet  Google Scholar 

  76. Verkijk, S., Roothaert, R., Pernisch, R., Schlobach, S.: Do you catch my drift? On the usage of embedding methods to measure concept shift in knowledge graphs. In: Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP’23, pp. 70–74. Association for Computing Machinery, New York, NY, USA (2023)

    Google Scholar 

  77. Vlontzos, A., Kainz, B., Gilligan-Lee, C.M.: Estimating categorical counterfactuals via deep twin networks. Nat. Mach. Intell. 5(2), 159–168 (2023)

    Article  Google Scholar 

  78. WBGU—German Advisory Council on Global Change. Humanity on the move: Unlocking the transformative power of cities. Technical report, WBGU, Berlin, 2016. Frauke Kraas, Claus Leggewie, Peter Lemke, Ellen Matthies, Dirk Messner, Nebojsa Nakicenovic, Hans Joachim Schellnhuber, Sabine Schlacke, Uwe Schneidewind

    Google Scholar 

  79. Yang, S., Yu, K., Cao, F., Liu, L., Wang, H., Li, J.: Learning causal representations for robust domain adaptation. IEEE Trans. Knowl. Data Eng. 35(3), 2750–2764, (2023). Conference Name: IEEE Transactions on Knowledge and Data Engineering

    Google Scholar 

  80. Yu, D., Yang, B., Liu, D., Wang, H., Pan, S.: A survey on neural-symbolic learning systems (2021)

    Google Scholar 

  81. Zhang, C., Zhang, K., Li, Y.: A causal view on robustness of neural networks. Adv. Neural Inf. Process. Syst. 33, 289–301 (2020)

    Google Scholar 

  82. Zhu, S., Ng, I., Chen, Z.: Causal discovery with reinforcement learning. In: International Conference on Learning Representations, Online, April 2020

    Google Scholar 

Download references

Acknowledgments

This research was partially supported by TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to André Meyer-Vitali .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Meyer-Vitali, A., Mulder, W. (2024). Engineering Principles for Building Trusted Human-AI Systems. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2024. Lecture Notes in Networks and Systems, vol 1066. Springer, Cham. https://doi.org/10.1007/978-3-031-66428-1_30

Download citation

Publish with us

Policies and ethics