research-article

Identifying Roles, Requirements and Responsibilities in Trustworthy AI Systems

Authors:

Will AbramsonAuthors Info & Claims

UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers

Pages 264 - 271

https://doi.org/10.1145/3460418.3479344

Published: 24 September 2021 Publication History

Abstract

Artificial Intelligence (AI) systems are being deployed around the globe in critical fields such as healthcare and education. In some cases, expert practitioners in these domains are being tasked with introducing or using such systems, but have little or no insight into what data these complex systems are based on, or how they are put together. In this paper, we consider an AI system from the domain practitioner’s perspective and identify key roles that are involved in system deployment. We consider the differing requirements and responsibilities of each role, and identify tensions between transparency and confidentiality that need to be addressed so that domain practitioners are able to intelligently assess whether a particular AI system is appropriate for use in their domain.

References

[1]

Ruha Benjamin. 2019. Race after technology: Abolitionist tools for the new jim code. Social Forces (2019).

[2]

Lyria Bennett Moses and Janet Chan. 2018. Algorithmic prediction in policing: assumptions, evaluation, and accountability. Policing and society 28, 7 (2018), 806–822.

[3]

Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José MF Moura, and Peter Eckersley. 2020. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 648–657.

Digital Library

[4]

Victoria R Brown and E Daly Vaughn. 2011. The writing on the (Facebook) wall: The use of social networking sites in hiring decisions. Journal of Business and psychology 26, 2 (2011), 219.

[5]

Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, 2020. Toward trustworthy AI development: mechanisms for supporting verifiable claims. arXiv preprint arXiv:2004.07213(2020).

[6]

Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.

[7]

J Adam Carter and Mona Simion. 2020. The ethics and epistemology of trust. Internet Encyclopedia of Philosophy(2020).

[8]

Enrico Coiera. 2019. The last mile: where artificial intelligence meets reality. Journal of medical Internet research 21, 11 (2019), e16323.

[9]

Fabiano Dalpiaz, Xavier Franch, and Jennifer Horkoff. 2016. istar 2.0 language guide. arXiv preprint arXiv:1605.07767(2016).

[10]

Andrew Davies, Tim Brady, and Michael Hobday. 2007. Organizing for solutions: Systems seller vs. systems integrator. Industrial marketing management 36, 2 (2007), 183–193.

[11]

Virginia Dignum, Melaniel Penagos, Klara Pigmans, and Steven Vosloo. 2020. Policy guidance on AI for children. (2020). https://www.unicef.org/globalinsight/reports/policy-guidance-ai-children

[12]

Florian Hawlitschek, Benedikt Notheisen, and Timm Teubner. 2018. The limits of trust-free systems: A literature review on blockchain technology and trust in the sharing economy. Electronic commerce research and applications 29 (2018), 50–63.

[13]

Anne Immonen, Marko Palviainen, and Eila Ovaska. 2014. Requirements of an open data based business ecosystem. IEEE access 2(2014), 88–103.

[14]

Azra Ismail and Neha Kumar. 2021. AI in Global Health: The View from the Front Lines. https://www.nehakumar.org/s/Ismail-Kumar-CHI-2021.pdf.

[15]

Vijay Khatri and Carol V Brown. 2010. Designing data governance. Commun. ACM 53, 1 (2010), 148–152.

Digital Library

[16]

PJ Nickel and K Vaesen. 2012. Risk and trust. Handbook of risk theory: epistemology, decision theory, ethics and social implications of risk(2012), 857–876.

[17]

Helen Nissenbaum. 2020. Privacy in context. Stanford University Press.

[18]

Emma Norris and Daryl B. O’Connor. 2019. Science as behaviour: Using a behaviour change approach to increase uptake of open science. Psychology & Health 34, 12 (2019), 1397–1406. https://doi.org/10.1080/08870446.2019.1679373

[19]

OECD. 2019. Artificial Intelligence in Society. https://doi.org/10.1787/eedfee77-en

[20]

Cathy O’Neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.

[21]

Onora O’Neill. 2002. A question of trust: The BBC Reith Lectures 2002. Cambridge University Press.

[22]

Alun Preece, Dan Harborne, Dave Braines, Richard Tomsett, and Supriyo Chakraborty. 2018. Stakeholders in Explainable AI. arXiv preprint arXiv:1810.00184(2018).

[23]

Cedric Renggli, Luka Rimanic, Nezihe Merve Gürel, Bojan Karlaš, Wentao Wu, and Ce Zhang. 2021. A Data Quality-Driven View of MLOps. arXiv preprint arXiv:2102.07750(2021).

[24]

Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[25]

Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552(2018).

[26]

Andrew Trask, Emma Bluemke, Ben Garfinkel, Claudia Ghezzou Cuervas-Mons, and Allan Dafoe. 2020. Beyond Privacy Trade-offs with Structured Transparency. arXiv preprint arXiv:2012.08347(2020).

[27]

Yanhao Wei, Pinar Yildirim, Christophe Van den Bulte, and Chrysanthos Dellarocas. 2015. Credit scoring with social network data. Marketing Science 35, 2 (2015), 234–258.

[28]

Eric Yu. 2011. Modeling Strategic Relationships for Process Reengineering.Social Modeling for Requirements Engineering 11, 2011(2011), 66–87.

Cited By

Lancaster CSchulenberg KFlathmann CMcNeese NFreeman G(2024)“It’s Everybody’s Role to Speak Up... But Not Everyone Will”: Understanding AI Professionals’ Perceptions of Accountability for AI Bias MitigationACM Journal on Responsible Computing10.1145/36321211:1(1-30)Online publication date: 20-Mar-2024
https://dl.acm.org/doi/10.1145/3632121
Wani NKumar RMamta Bedi JRida I(2024)Explainable AI-driven IoMT fusion: Unravelling techniques, opportunities, and challenges with Explainable AI in healthcareInformation Fusion10.1016/j.inffus.2024.102472110(102472)Online publication date: Oct-2024
https://doi.org/10.1016/j.inffus.2024.102472
Franke U(2024)The Limits of Calibration and the Possibility of Roles for Trustworthy AIPhilosophy & Technology10.1007/s13347-024-00771-737:3Online publication date: 1-Jul-2024
https://doi.org/10.1007/s13347-024-00771-7
Show More Cited By

Recommendations

The Ethics of AI Ethics: An Evaluation of Guidelines
Abstract
Current advances in research, development and application of artificial intelligence (AI) systems have yielded a far-reaching discourse on AI ethics. In consequence, a number of ethics guidelines have been released in recent years. These ...
Towards ethical aspects on artificial intelligence
AIKED'09: Proceedings of the 8th WSEAS international conference on Artificial intelligence, knowledge engineering and data bases

This paper presents the role of ethics in developing artificial intelligence, and how the artificial intelligence could change our perspective, because artificial intelligence in fact is all around us. Artificial intelligence is an important part of our ...
Ethical artificial intelligence framework for a good AI society: principles, opportunities and perils
Abstract
The justification and rationality of this paper is to present some fundamental principles, theories, and concepts that we believe moulds the nucleus of a good artificial intelligence (AI) society. The morally accepted significance and utilitarian ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers

September 2021

711 pages

ISBN:9781450384612

DOI:10.1145/3460418

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

UbiComp '21

Sponsor:

UbiComp '21: The 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 21 - 26, 2021

Virtual, USA

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
263
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)12

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lancaster CSchulenberg KFlathmann CMcNeese NFreeman G(2024)“It’s Everybody’s Role to Speak Up... But Not Everyone Will”: Understanding AI Professionals’ Perceptions of Accountability for AI Bias MitigationACM Journal on Responsible Computing10.1145/36321211:1(1-30)Online publication date: 20-Mar-2024
https://dl.acm.org/doi/10.1145/3632121
Wani NKumar RMamta Bedi JRida I(2024)Explainable AI-driven IoMT fusion: Unravelling techniques, opportunities, and challenges with Explainable AI in healthcareInformation Fusion10.1016/j.inffus.2024.102472110(102472)Online publication date: Oct-2024
https://doi.org/10.1016/j.inffus.2024.102472
Franke U(2024)The Limits of Calibration and the Possibility of Roles for Trustworthy AIPhilosophy & Technology10.1007/s13347-024-00771-737:3Online publication date: 1-Jul-2024
https://doi.org/10.1007/s13347-024-00771-7
Westerstrand S(2024)Reconstructing AI Ethics Principles: Rawlsian Ethics of Artificial IntelligenceScience and Engineering Ethics10.1007/s11948-024-00507-y30:5Online publication date: 9-Oct-2024
https://doi.org/10.1007/s11948-024-00507-y
Novelli CTaddeo MFloridi L(2024)Accountability in artificial intelligence: what it is and how it worksAI & Society10.1007/s00146-023-01635-y39:4(1871-1882)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00146-023-01635-y
Franke UHelgesson Hallström CArtman HDexe J(2024)Requirements on and Procurement of Explainable Algorithms—A Systematic Review of the LiteratureNew Trends in Disruptive Technologies, Tech Ethics, and Artificial Intelligence10.1007/978-3-031-66635-3_4(40-52)Online publication date: 13-Aug-2024
https://doi.org/10.1007/978-3-031-66635-3_4
Harb WQabajeh M(2024)Impact of Generative Artificial Intelligence on Journalism: Practice and DeontologyJournalism, Digital Media and the Fourth Industrial Revolution10.1007/978-3-031-63153-5_18(241-255)Online publication date: 4-Sep-2024
https://doi.org/10.1007/978-3-031-63153-5_18
Nweke LYayilgan S(2024)Opportunities and Challenges of Using Artificial Intelligence in Securing Cyber-Physical SystemsArtificial Intelligence for Security10.1007/978-3-031-57452-8_7(131-164)Online publication date: 17-Apr-2024
https://doi.org/10.1007/978-3-031-57452-8_7
Albahri ADuhaim AFadhel MAlnoor ABaqer NAlzubaidi LAlbahri OAlamoodi ABai JSalhi ASantamaría JOuyang CGupta AGu YDeveci M(2023)A systematic review of trustworthy and explainable artificial intelligence in healthcareInformation Fusion10.1016/j.inffus.2023.03.00896:C(156-191)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1016/j.inffus.2023.03.008
Klenk M(2023)Algorithmic Transparency and ManipulationPhilosophy & Technology10.1007/s13347-023-00678-936:4Online publication date: 15-Dec-2023
https://doi.org/10.1007/s13347-023-00678-9

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents