research-article

Building Jiminy Cricket: An Architecture for Moral Agreements Among Stakeholders

Authors:

Marija Slavkovik,

Leendert van der TorreAuthors Info & Claims

AIES '19: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society

Pages 147 - 153

https://doi.org/10.1145/3306618.3314257

Published: 27 January 2019 Publication History

Abstract

An autonomous system is constructed by a manufacturer, operates in a society subject to norms and laws, and is interacting with end-users. We address the challenge of how the moral values and views of all stakeholders can be integrated and reflected in the moral behavior of the autonomous system. We propose an artificial moral agent architecture that uses techniques from normative systems and formal argumentation to reach moral agreements among stakeholders. We show how our architecture can be used not only for ethical practical reasoning and collaborative decision-making, but also for the explanation of such moral behavior.

References

[1]

C. E. Alchourron. 1991. Conflcits of norms and revision of normative systems. Law and Philosophy, Vol. 10 (1991), 413--425.

[2]

M. Anderson and S. Leigh Anderson. 2014. GenEth: A General Ethical Dilemma Analyzer. In Proceedings of the 28th AAAI Conference on AI. 253--261. http://www.aaai.org/ocs/index.php/AAAI/AAAI14/paper/view/8308

Digital Library

[3]

R.C. Arkin, P. Ulam, and A. R. Wagner. 2012. Moral Decision Making in Autonomous Systems: Enforcement, Moral Emotions, Dignity, Trust, and Deception. Proc. of the IEEE, Vol. 100, 3 (2012), 571--589.

[4]

P. Baroni, D. Gabbay, M. Giacomin, and L. van der Torre (Eds.). 2018. Handbook of Formal Argumentation .College Publications.

[5]

T. Bench-Capon, K. Atkinson, and A. Chorley. 2005. Persuasion and Value in Legal Argument. Journal of Logic and Computation Comput., Vol. 15, 6 (2005), 1075--1097.

Digital Library

[6]

O. Biran and C. Cotton. 2017. Explanation and Justification in Machine Learning: A Survey. In Proceedings of the IJCAI Workshop on Explainable Artificial Intelligence (XAI 2017). 8--13.

[7]

R. Booth, M. Caminada, and B. Marshall. 2018. DISCO: A Web-Based Implementation of Discussion Games for Grounded and Preferred Semantics. In COMMA (Frontiers in Artificial Intelligence and Applications), Vol. 305. IOS Press, 453--454.

[8]

S. Bringsjord, K. Arkoudas, and P. Bello. 2008. Toward a general logicist methodology for engineering ethically correct robots. IEEE Intelligent Systems, Vol. 21, 4 (2008), 38--44.

Digital Library

[9]

V. Charisi, L.A. Dennis, M. Fisher, R. Lieck, A. Matthias, M. Slavkovik, J. Sombetzki, A.F.T. Winfield, and R. Yampolskiy. 2017. Towards Moral Autonomous Systems. CoRR, Vol. abs/1703.04741 (2017). http://arxiv.org/abs/1703.04741

[10]

A. Chopra, L. van der Torre, H. Verhagen, and S. Villata. 2018. Handbook of normative multiagent systems .College Publications.

[11]

O. Cocarascu, K. vCyras, and F. Toni. 2018. Explanatory Predictions with Artificial Neural Networks and Argumentation. In Proceedings of the IJCAI/ECAI Workshop on Explainable Artificial Intelligence (XAI 2018). 26--32.

[12]

L. A. Dennis, M. Fisher, M. Slavkovik, and M. P. Webster. 2016. Formal Verification of Ethical Choices in Autonomous Systems. Robotics and Autonomous Systems, Vol. 77 (2016), 1--14.

Digital Library

[13]

L. A. Dennis, M. Fisher, and A. F. T. Winfield. 2015. Towards Verifiably Ethical Robot Behaviour. In Proceedings of AAAI Workshop on AI and Ethics. http://aaai.org/ocs/index.php/WS/AAAIW15/paper/view/10119.

[14]

V. Dignum. 2017. Responsible Autonomy. In Proceedings of the 26th IIJCAI. 4698--4704.

Digital Library

[15]

P. M. Dung. 1995. On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games. Artificial Intelligence, Vol. 77, 2 (1995), 321--358.

Digital Library

[16]

S. Dyrkolbotn, T. Pedersen, and M. Slavkovik. 2018. On the distinction between implicit and explicit ethical agency. In AAAI/ACM AIES conference. New Orleans, USA.

Digital Library

[17]

A. Etzioni and O. Etzioni. 2017. Incorporating Ethics into Artificial Intelligence. The Journal of Ethics (2017), 1--16.

[18]

International Society of Automotive Engineers. 2016. September 2016, Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. http://standards.sae.org/j3016_201609/.

[19]

F. Lindner and M.M. Bentzen. 2017. The Hybrid Ethical Reasoning Agent IMMANUEL. In Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, HRI 2017, Vienna, Austria, March 6--9, 2017. 187--188.

Digital Library

[20]

B. F. Malle, M. Scheutz, T. Arnold, J. Voiklis, and C. Cusimano. 2015. Sacrifice One For the Good of Many?: People Apply Different Moral Norms to Human and Robot Agents. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI '15). ACM, 117--124.

Digital Library

[21]

T. Miller. 2017. Explanation in Artificial Intelligence: Insights from the Social Sciences. CoRR, Vol. abs/1706.07269 (2017). arxiv: 1706.07269 http://arxiv.org/abs/1706.07269

[22]

S. Modgil and H. Prakken. 2013. A general account of argumentation with preferences. Artificial Intelligence, Vol. 195 (2013), 361--397.

Digital Library

[23]

J. H. Moor. 2006. The Nature, Importance, and Difficulty of Machine Ethics. IEEE Intelligent Systems, Vol. 21, 4 (July 2006), 18--21.

Digital Library

[24]

E. Pires Bjørgen, S. Øvervatn Madsen, T. Skaar Bjørknes, F. Vonheim Heimsæter, R. Håvik, M. Linderud, P.N. Longberg, L.A. Dennis, and M. Slavkovik. 2018. Cake, death, and trolleys: dilemmas as benchmarks of ethical decision-making. In AAAI/ACM AIES Conference. New Orleans, USA.

Digital Library

[25]

A. Sergeant. 2013. Automatic Argumentation Extraction. In The Semantic Web: Semantics and Big Data, P. Cimiano, O. Corcho, V. Presutti, L. Hollink, and S. Rudolph (Eds.). Springer ß, Berlin, Heidelberg, 656--660.

[26]

D. Vanderelst and A. Winfield. 2017. An architecture for ethical robots inspired by the simulation theory of cognition. Cognitive Systems Research (2017).

[27]

K. vCyras, K. Satoh, and F. Toni. 2016. Explanation for Case-Based Reasoning via Abstract Argumentation. In Computational Models of Argument - Proceedings of COMMA. 243--254.

[28]

G. Vreeswijk and H. Prakken. 2000. Credulous and Sceptical Argument Games for Preferred Semantics. In JELIA (LNCS), Vol. 1919. Springer, 239--253.

Digital Library

[29]

W. Wallach and C. Allen. 2008. Moral Machines: Teaching Robots Right from Wrong .Oxford University Press.

Digital Library

[30]

D. Walton. 2011. A dialogue system specification for explanation. Synthese, Vol. 182, 3 (2011), 349--374.

[31]

D. Walton. 2012. Using Argumentation Schemes for Argument Extraction: A Bottom-Up Method. Internat. Journal of Cognitive Informatics and Natural Intelligence, Vol. 3 (2012).

[32]

P. Ziafati. 2015. Information Engineering in Autonomous Robot Software. Ph.D. Dissertation. University of Luxembourg.

Cited By

Kampik TNieves J(2025)Disagree and commit: degrees of argumentation-based agreementsAutonomous Agents and Multi-Agent Systems10.1007/s10458-025-09688-739:1Online publication date: 27-Jan-2025
https://doi.org/10.1007/s10458-025-09688-7
Memon MScoccia GAutili MInverardi P(2024)An Architecture for Ethics-Based Negotiation in the Decision-Making of Intelligent Autonomous Systems2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C)10.1109/ICSA-C63560.2024.00016(60-64)Online publication date: 4-Jun-2024
https://doi.org/10.1109/ICSA-C63560.2024.00016
Ozaki ARehman ASlavkovik M(2024)Finding middle grounds for incoherent horn expressions: the moral machine caseAutonomous Agents and Multi-Agent Systems10.1007/s10458-024-09681-638:2Online publication date: 16-Oct-2024
https://dl.acm.org/doi/10.1007/s10458-024-09681-6
Show More Cited By

Index Terms

Building Jiminy Cricket: An Architecture for Moral Agreements Among Stakeholders

Recommendations

Robots: ethical by design

Among ethicists and engineers within robotics there is an ongoing discussion as to whether ethical robots are possible or even desirable. We answer both of these questions in the positive, based on an extensive literature study of existing arguments. ...
Artificial systems with moral capacities? A research design and its implementation in a geriatric care system
Abstract
The development of increasingly intelligent and autonomous technologies will eventually lead to these systems having to face morally problematic situations. This gave rise to the development of artificial morality, an emerging field in ...
Artificial virtuous agents in a multi-agent tragedy of the commons
Abstract
Although virtue ethics has repeatedly been proposed as a suitable framework for the development of artificial moral agents (AMAs), it has been proven difficult to approach from a computational perspective. In this work, we present the first ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AIES '19: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society

January 2019

577 pages

ISBN:9781450363242

DOI:10.1145/3306618

General Chairs:
Vincent Conitzer
Duke University, USA
,
Gillian Hadfield
University of Toronto + Vector Institute, Canada; OpenAI, USA
,
Shannon Vallor
Santa Clara University, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
AAAI: American Association for Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Fundamental Research Funds for the Central Universities of China for the project Big Data Reasoning and Decision Making
National Social Science Foundation of China

Conference

AIES '19

Sponsor:

SIGAI
AAAI

AIES '19: AAAI/ACM Conference on AI, Ethics, and Society

January 27 - 28, 2019

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 61 of 162 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
442
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kampik TNieves J(2025)Disagree and commit: degrees of argumentation-based agreementsAutonomous Agents and Multi-Agent Systems10.1007/s10458-025-09688-739:1Online publication date: 27-Jan-2025
https://doi.org/10.1007/s10458-025-09688-7
Memon MScoccia GAutili MInverardi P(2024)An Architecture for Ethics-Based Negotiation in the Decision-Making of Intelligent Autonomous Systems2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C)10.1109/ICSA-C63560.2024.00016(60-64)Online publication date: 4-Jun-2024
https://doi.org/10.1109/ICSA-C63560.2024.00016
Ozaki ARehman ASlavkovik M(2024)Finding middle grounds for incoherent horn expressions: the moral machine caseAutonomous Agents and Multi-Agent Systems10.1007/s10458-024-09681-638:2Online publication date: 16-Oct-2024
https://dl.acm.org/doi/10.1007/s10458-024-09681-6
Buraglio GDvořák WRapberger AWoltran S(2024)Constrained Derivation in Assumption-Based ArgumentationFoundations of Information and Knowledge Systems10.1007/978-3-031-56940-1_19(340-359)Online publication date: 29-Mar-2024
https://doi.org/10.1007/978-3-031-56940-1_19
Malle BRosen EChi VRamesh D(2023)What Properties of Norms can we Implement in Robots?*2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309355(2598-2603)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309355
Yu ZJu SChen W(2023)Context-based argumentation frameworks and multi-agent consensus buildingJournal of Logic and Computation10.1093/logcom/exad063Online publication date: 19-Oct-2023
https://doi.org/10.1093/logcom/exad063
Li CLiao B(2023)Integrating individual preferences into collective argumentationJournal of Logic and Computation10.1093/logcom/exac09733:2(344-369)Online publication date: 19-Jan-2023
https://doi.org/10.1093/logcom/exac097
Al Anaissy CVesic SNevejans N(2023)Towards Ethical Argumentative Persuasive ChatbotsCoordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XVI10.1007/978-3-031-49133-7_8(141-160)Online publication date: 20-Dec-2023
https://doi.org/10.1007/978-3-031-49133-7_8
Memon M(2023)A Brief Overview of an Approach Towards Ethical Decision-MakingMulti-Agent Systems10.1007/978-3-031-43264-4_32(458-464)Online publication date: 7-Sep-2023
https://doi.org/10.1007/978-3-031-43264-4_32
Beishui L(2022)On Interdisciplinary Studies of a New Generation of Artificial Intelligence and LogicSocial Sciences in China10.1080/02529203.2022.212220743:3(21-42)Online publication date: 30-Sep-2022
https://doi.org/10.1080/02529203.2022.2122207
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten