research-article

Open access

RE-centric Recommendations for the Development of Trustworthy(er) Autonomous Systems

Authors:

Krishna Ronanki,

Beatriz Cabrero-Daniel,

Jennifer Horkoff,

Christian BergerAuthors Info & Claims

TAS '23: Proceedings of the First International Symposium on Trustworthy Autonomous Systems

Article No.: 1, Pages 1 - 8

https://doi.org/10.1145/3597512.3599697

Published: 11 July 2023 Publication History

All formats PDF

Abstract

Complying with the EU AI Act (AIA) guidelines while developing and implementing AI systems will soon be mandatory within the EU. However, practitioners lack actionable instructions to operationalise ethics during AI systems development. A literature review of different ethical guidelines revealed inconsistencies in the principles addressed and the terminology used to describe them. Furthermore, requirements engineering (RE), which is identified to foster trustworthiness in the AI development process from the early stages was observed to be absent in a lot of frameworks that support the development of ethical and trustworthy AI. This incongruous phrasing combined with a lack of concrete development practices makes trustworthy AI development harder. To address these concerns, we formulated a comparison table for the terminology used and the coverage of the ethical AI principles in major ethical AI guidelines. We then examined the applicability of ethical AI development frameworks for performing effective RE during the development of trustworthy AI systems. A tertiary review and meta-analysis of literature discussing ethical AI frameworks revealed their limitations when developing trustworthy AI. Based on our findings, we propose recommendations to address such limitations during the development of trustworthy AI.

References

[1]

[n. d.]. Translation: Measures for the Management of Generative Artificial Intelligence Services (Draft for Comment) – April 2023. https://digichina.stanford.edu/work/translation-measures-for-the-management-of-generative-artificial-intelligence-services-draft-for-comment-april-2023/

[2]

Glenda Amaral, Renata Guizzardi, Giancarlo Guizzardi, and John Mylopoulos. 2021. Trustworthiness requirements: the pix case study. In Conceptual Modeling: 40th International Conference, ER 2021, Virtual Event, October 18–21, 2021, Proceedings 40. Springer, 257–267.

Digital Library

[3]

Fatma Başak Aydemir and Fabiano Dalpiaz. 2018. A roadmap for ethics-aware software engineering. In Proceedings of the International Workshop on Software Fairness. 15–21.

Digital Library

[4]

Josef Baker-Brunnbauer. 2021. TAII framework for trustworthy AI systems. Baker-Brunnbauer, J.(2021). TAII Framework for Trustworthy AI Systems. ROBONOMICS: The Journal of the Automated Economy 2 (2021), 17.

[5]

Brian Berenbach, Daniel J Paulish, Juergen Kazmeier, and Arnold Rudorfer. 2009. Software & systems requirements engineering: in practice. McGraw-Hill Education.

[6]

Lindsey Brodie and Mark Woodman. 2011. Prioritization of stakeholder value using metrics. In Evaluation of Novel Approaches to Software Engineering: 5th International Conference, ENASE 2010, Athens, Greece, July 22-24, 2010, Revised Selected Papers 5. Springer, 74–88.

[7]

Angela Campbell and Kathleen Cranley Glass. 2000. The legal status of clinical and ethics policies, codes, and guidelines in medical practice and research. McGill LJ 46 (2000), 473.

[8]

Cansu Canca. 2020. Operationalizing AI Ethics Principles. Commun. ACM 63, 12 (nov 2020), 18–21. https://doi.org/10.1145/3430368

Digital Library

[9]

Simon Chesterman. 2020. Artificial intelligence and the problem of autonomy. Notre Dame J. on Emerging Tech. 1 (2020), 210.

[10]

James Crowley, AP OrSullivan, Andrzej Nowak, Catholijn Jonker, Dino Pedreschi, Fosca Giannotti, and Y Rogers. 2019. Toward ai systems that augment and empower humans by understanding us, our society and the world around us. Report of 761758 (2019), 1–32.

[11]

Francien Dechesne. 2020. Requirements engineering for moral considerations in algorithmic systems : RE’20 Conference Keynote. In 2020 IEEE 28th International Requirements Engineering Conference (RE). 1–2. https://doi.org/10.1109/RE48521.2020.00010

[12]

Virginia Dignum and CO Confidential. 2019. HumanE AI. (2019).

[13]

Yogesh K Dwivedi, Laurie Hughes, Elvira Ismagilova, Gert Aarts, Crispin Coombs, Tom Crick, Yanqing Duan, Rohita Dwivedi, John Edwards, Aled Eirug, 2021. Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management 57 (2021), 101994.

[14]

Hallensleben et al. 2020. Artificial Intelligence Ethics Impact Group AIEI Group. From Principles to Practice. An Interdisciplinary Framework to Operationalise AI Ethics. https://www.ai-ethics-impact.org/resource/blob/1961130/c6db9894ee73aefa489d6249f5ee2b9f/aieig—report—download-hb-data.pdf

[15]

Content European Commission, Directorate-General for Communications Networks and Technology. 2021. EUR-Lex - 52021PC0206 - EN - EUR-Lex. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206

[16]

David Fernández Llorca and Emilia Gómez. 2021. Trustworthy autonomous vehicles. Publications Office of the European Union, Luxembourg, EUR 30942 (2021). https://doi.org/doi/10.2760/120385

[17]

Luciano Floridi, Matthias Holweg, Mariarosaria Taddeo, Javier Amaya Silva, Jakob Mökander, and Yuni Wen. 2022. CapAI-A Procedure for Conducting Conformity Assessment of AI Systems in Line with the EU Artificial Intelligence Act. Available at SSRN 4064091 (2022).

[18]

Hans-Martin Heyn, Eric Knauss, Iswarya Malleswaran, and Shruthi Dinakaran. 2023. An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications. arXiv preprint arXiv:2301.13476 (2023).

[19]

Hans-Martin Heyn, Eric Knauss, and Patrizio Pelliccione. 2023. A compositional approach to creating architecture frameworks with an application to distributed AI systems. Journal of Systems and Software (2023), 111604.

[20]

IEEE. 2021. The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, Version 2. IEEE, 2017. IEEE. http://standards.ieee.org/develop/indconn/ec/autonomous_systems.html

[21]

Expert Group On How AI Principles Should Be Implemented. 2021. AI Governance in Japan Ver. 1.1. (2021). https://www.meti.go.jp/english/press/2020/0616_001.html.

[22]

Lakshmi Shankar Iyer. 2021. AI enabled applications towards intelligent transportation. Transportation Engineering 5 (2021), 100083.

[23]

Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature Machine Intelligence 1, 9 (2019), 389–399.

[24]

Amruta Kale, Tin Nguyen, Frederick C Harris Jr, Chenhao Li, Jiyin Zhang, and Xiaogang Ma. 2022. Provenance documentation to enable explainable and trustworthy AI: A literature review. Data Intelligence (2022), 1–41.

[25]

Davinder Kaur, Suleyman Uslu, Kaley J Rittichier, and Arjan Durresi. 2022. Trustworthy artificial intelligence: a review. ACM Computing Surveys (CSUR) 55, 2 (2022), 1–38.

Digital Library

[26]

Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering. 2 (01 2007).

[27]

Blagovesta Kostova, Seda Gurses, and Alain Wegmann. 2020. On the Interplay between Requirements, Engineering, and Artificial Intelligence. In REFSQ Workshops.

[28]

Brent Mittelstadt. 2019. Principles alone cannot guarantee ethical AI. Nature Machine Intelligence 1, 11 (2019), 501–507.

[29]

Jessica Morley, Luciano Floridi, Libby Kinsey, and Anat Elhalal. 2020. From what to how: an initial review of publicly available AI ethics tools, methods and research to translate principles into practices. Science and engineering ethics 26, 4 (2020), 2141–2168.

[30]

OECD. 2019. Artificial Intelligence in Society. OECD. https://doi.org/10.1787/eedfee77-en

[31]

High-Level Expert Group on Artificial Intelligence. 2019. Ethics Guidelines for Trustworthy Artificial Intelligence (AI). www.aepd.es/sites/default/files/2019-12/ai-ethics-guidelines.pdf

[32]

Erich Prem. 2023. From ethical AI frameworks to tools: a review of approaches. AI and Ethics (2023), 1–18.

[33]

UK research and innovation. 2019. responsible research and innovation (RRI) framework. https://www.ukri.org/councils/epsrc/guidance-for-applicants/what-to-include-in-your-proposal/health-technologies-impact-and-translation-toolkit/research-integrity-in-healthcare-technologies/responsible-research-and-innovation/

[34]

Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering. Empirical software engineering 14 (2009), 131–164.

[35]

Ahmed Abdo Ali Saeed and Seok-Won Lee. 2018. Non-functional requirements trade-off in self-adaptive systems. In 2018 4th International Workshop on Requirements Engineering for Self-Adaptive, Collaborative, and Cyber Physical Systems (RESACS). IEEE, 9–15.

[36]

Joseph Sifakis and David Harel. 2022. Trustworthy Autonomous System Development. ACM Transactions on Embedded Computing Systems (2022).

[37]

Lorne Sossin and Charles W Smith. 2002. Hard choices and soft law: ethical codes, policy guidelines and the role of the courts in regulating government. Alta. L. Rev. 40 (2002), 867.

[38]

Scott Thiebes, Sebastian Lins, and Ali Sunyaev. 2021. Trustworthy artificial intelligence. Electronic Markets 31 (2021), 447–464.

[39]

UNESCO. 2022. Recommendation on the Ethics of Artificial Intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000381137

[40]

Ville Vakkuri, Kai-Kristian Kemell, and Pekka Abrahamsson. 2019. Implementing ethics in AI: initial results of an industrial multiple case study. In Product-Focused Software Process Improvement: 20th International Conference, PROFES 2019, Barcelona, Spain, November 27–29, 2019, Proceedings 20. Springer, 331–338.

[41]

Ville Vakkuri, Kai-Kristian Kemell, Marianna Jantunen, Erika Halme, and Pekka Abrahamsson. 2021. ECCOLA—A method for implementing ethically aligned AI systems. Journal of Systems and Software 182 (2021), 111067.

Digital Library

[42]

Karl Werder, Balasubramaniam Ramesh, and Rongen Zhang. 2022. Establishing data provenance for responsible artificial intelligence systems. ACM Transactions on Management Information Systems (TMIS) 13, 2 (2022), 1–23.

Digital Library

[43]

Karl Wiegers and Joy Beatty. 2013. Software requirements. Pearson Education.

[44]

Jianlong Zhou, Fang Chen, Adam Berry, Mike Reed, Shujia Zhang, and Siobhan Savage. 2020. A Survey on Ethical Principles of AI and Implementations. 2020 IEEE Symposium Series on Computational Intelligence, SSCI 2020 (12 2020), 3010–3017. https://doi.org/10.1109/SSCI47803.2020.9308437

Cited By

Gupta BGaurav AChui KPsannis K(2024)The Future of Ethical AI in Large Language ModelsChallenges in Large Language Model Development and AI Ethics10.4018/979-8-3693-3860-5.ch013(410-435)Online publication date: 30-Aug-2024
https://doi.org/10.4018/979-8-3693-3860-5.ch013
Gorzny JDerka M(2024)Requirements Engineering Challenges for Blockchain Rollups2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00052(340-347)Online publication date: 24-Jun-2024
https://doi.org/10.1109/REW61692.2024.00052

Index Terms

RE-centric Recommendations for the Development of Trustworthy(er) Autonomous Systems
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Towards an AI-Centric Requirements Engineering Framework for Trustworthy AI
ICSE '23: Proceedings of the 45th International Conference on Software Engineering: Companion Proceedings

Ethical guidelines are an asset for artificial intelligence(AI) development and conforming to them will soon be a procedural requirement once the EU AI Act gets ratified in the European parliament. However, developers often lack explicit knowledge on ...
Categorical Artificial Intelligence: The Integration of Symbolic and Statistical AI for Verifiable, Ethical, and Trustworthy AI
Artificial General Intelligence
Abstract
Statistical artificial intelligence based upon machine learning is facing major challenges such as machine bias, explainability, and verifiability problems. Resolving them would be of the utmost importance for ethical, safe, and responsible AI for ...
Aspects and Views on Responsible Artificial Intelligence
Machine Learning, Optimization, and Data Science
Abstract
Background: There is a lot of discussion in EU politics about trust in artificial intelligence (AI). Because it can be used as a lethal weapon we need (EU) regulations that take care of setting up a framework. Companies need guidance to develop ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

TAS '23: Proceedings of the First International Symposium on Trustworthy Autonomous Systems

July 2023

426 pages

ISBN:9798400707346

DOI:10.1145/3597512

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

TAS '23

TAS '23: First International Symposium on Trustworthy Autonomous Systems

July 11 - 12, 2023

Edinburgh, United Kingdom

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
518
Total Downloads

Downloads (Last 12 months)415
Downloads (Last 6 weeks)59

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gupta BGaurav AChui KPsannis K(2024)The Future of Ethical AI in Large Language ModelsChallenges in Large Language Model Development and AI Ethics10.4018/979-8-3693-3860-5.ch013(410-435)Online publication date: 30-Aug-2024
https://doi.org/10.4018/979-8-3693-3860-5.ch013
Gorzny JDerka M(2024)Requirements Engineering Challenges for Blockchain Rollups2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00052(340-347)Online publication date: 24-Jun-2024
https://doi.org/10.1109/REW61692.2024.00052

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten