research-article

Implementations in Machine Ethics: A Survey

Authors:

Suzanne Tolmeijer,

Cristina Sarasua,

Markus Christen,

Abraham BernsteinAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 53, Issue 6

Article No.: 132, Pages 1 - 38

https://doi.org/10.1145/3419633

Published: 30 December 2020 Publication History

Abstract

Increasingly complex and autonomous systems require machine ethics to maximize the benefits and minimize the risks to society arising from the new technology. It is challenging to decide which type of ethical theory to employ and how to implement it effectively. This survey provides a threefold contribution. First, it introduces a trimorphic taxonomy to analyze machine ethics implementations with respect to their object (ethical theories), as well as their nontechnical and technical aspects. Second, an exhaustive selection and description of relevant works is presented. Third, applying the new taxonomy to the selected works, dominant research patterns, and lessons for the field are identified, and future directions for research are suggested.

References

[1]

David Abel, James MacGlashan, and Michael L. Littman. 2016. Reinforcement learning as a framework for ethical decision making. In Proceedings of the AAAI Workshop: AI, Ethics, and Society, Vol. 16. AAAI Press, Palo Alto, CA, 02.

[2]

Colin Allen, Iva Smit, and Wendell Wallach. 2005. Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics Inf. Technol. 7, 3 (2005), 149--155.

Digital Library

[3]

Michael Anderson and Susan Leigh Anderson. 2008. EthEl: Toward a principled ethical eldercare robot. In Proceedings of the AAAI Fall 2008 Symposium on AI in Eldercare: New Solutions to Old Problems. AAAI Press, Palo Alto, CA, 4--11.

[4]

Michael Anderson and Susan Leigh Anderson. 2010. Robot be good. Sci. Am. 303, 4 (2010), 72--77.

[5]

Michael Anderson and Susan Leigh Anderson. 2018. GenEth: A general ethical dilemma analyzer. J. Behavi. Robot. 9, 1 (2018), 337--357.

[6]

Michael Anderson, Susan Leigh Anderson, and Chris Armen. 2004. Towards machine ethics. In Proceedings of the AAAI-04 Workshop on Agent Publishers: Theory and Practice. AAAI Press, Palo Alto, CA, 2--7.

[7]

Michael Anderson, Susan Leigh Anderson, and Chris Armen. 2006. MedEthEx: A prototype medical ethics advisor. In Proceedings of the 18th Conference on Innovative Applications of Artificial Intelligence, Volume 2 (IAAI’06). AAAI Press, Palo Alto, CA, 1759--1765.

[8]

M. Anderson, S. L. Anderson, and V. Berenz. 2019. A value-driven eldercare robot: Virtual and physical instantiations of a case-supported principle-based behavior paradigm. Proc. IEEE 107, 3 (2019), 526--540.

[9]

R. C. Arkin. 2007. Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture. GVU Technical Report GIT-GVU-07-11, S. 1--117. Georgia Institute of Technology.

[10]

R. C. Arkin. 2008. Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture-Part III: Representational and Architectural Considerations. Technical Report. Georgia Institute of Technology.

[11]

R. C. Arkin. 2008. Governing ethical behavior: Embedding an ethical controller in a hybrid deliberative-reactive robot architecture—Part II: Formalization for ethical control. In Proceedings of the 1st Artificial General Intelligence Conference 2008. IOS Press, Amsterdam, The Netherlands.

[12]

R. C. Arkin. 2008. Governing lethal behavior: Embedding ethics in a hybrid deliberative/reactive robot architecture part I: Motivation and philosophy. In Proceedings of the 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI’08). ACM, New York, NY, 121--128.

[13]

Konstantine Arkoudas, Selmer Bringsjord, and Paul Bello. 2005. Toward ethical robots via mechanized deontic logic. In Proceedings of the AAAI Fall Symposium on Machine Ethics. AAAI Press, Menlo Park, CA, 17--23.

[14]

Stuart Armstrong. 2015. Motivated value selection for artificial agents. In Proceedings of the AAAI Workshop: AI and Ethics, Vol. 92. AAAI Press, Palo Alto, CA, 12--20.

[15]

Kevin D. Ashley and Bruce M. McLaren. 1994. A CBR knowledge representation for practical ethics. In Proceedings of the European Workshop on Advances in Case-Based Reasoning. Springer, Berlin, 180--197.

[16]

Katie Atkinson and Trevor Bench-Capon. 2007. Action-based alternating transition systems for arguments about action. In Proceedings of the 22nd AAAI Conference on Artificial Intelligence, Vol. 7. AAAI Press, Palo Alto, CA, 24--29.

[17]

Katie Atkinson and Trevor Bench-Capon. 2008. Addressing moral problems through practical reasoning. J. Appl. Logic 6, 2 (2008), 135--151.

[18]

Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-François Bonnefon, and Iyad Rahwan. 2018. The moral machine experiment. Nature 563, 7729 (2018), 59.

[19]

Meisam Azad-Manjiri. 2014. A new architecture for making moral agents based on C4. 5 decision tree algorithm. Int. J. Inf. Technol. Comput. Sci. 6, 5 (2014), 50.

[20]

Chitta Baral. 2003. Knowledge Representation, Reasoning and Declarative Problem Solving. Cambridge University Press, Cambridge, UK.

Digital Library

[21]

Seth D. Baum. 2020. Social choice ethics in artificial intelligence. AI 8 Soc. 35 (2020), 165--176. https://doi.org/10.1007/s00146-017-0760-1.

[22]

Tom L. Beauchamp and James F. Childress. 2001. Principles of Biomedical Ethics. Oxford University Press, New York, NY.

[23]

T. J. M. Bench-Capon. 2020. Ethical approaches and autonomous systems. Artif. Intell. 281 (2020), 103239.

[24]

Trevor Bench-Capon, Katie Atkinson, and Alison Chorley. 2005. Persuasion and value in legal argument. J. Logic Comput. 15, 6 (2005), 1075--1097.

Digital Library

[25]

Trevor Bench-Capon and Giovanni Sartor. 2003. A model of legal reasoning with cases incorporating theories and values. Artif. Intell. 150, 1-2 (2003), 97--143.

Digital Library

[26]

Oliver Bendel. 2019. Handbuch Maschinenethik. Springer.

[27]

Fiona Berreby, Gauvain Bourgne, and Jean-Gabriel Ganascia. 2015. Modelling moral reasoning and ethical responsibility with logic programming. In Logic for Programming, Artificial Intelligence, and Reasoning. Springer, Berlin, 532--548.

[28]

Fiona Berreby, Gauvain Bourgne, and Jean-Gabriel Ganascia. 2017. A declarative modular framework for representing and applying ethical principles. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 96--104.

[29]

Simon Blackburn. 2002. Being Good: A Short Introduction to Ethics. Oxford University Press, Oxford, UK.

[30]

Simon Blackburn. 2016. The Oxford Dictionary of Philosophy. Oxford University Press, Oxford, UK.

[31]

Jean-François Bonnefon, Azim Shariff, and Iyad Rahwan. 2016. The social dilemma of autonomous vehicles. Science 352, 6293 (2016), 1573--1576.

[32]

Vincent Bonnemains, Claire Saurel, and Catherine Tessier. 2018. Embedded ethics: Some technical and ethical challenges. Ethics Inf. Technol. 20, 1 (2018), 41--58.

[33]

N. Bostrom. 2014. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, Oxford, UK.

[34]

David Bourget and David J. Chalmers. 2014. What do philosophers believe? Philos. Stud. 170, 3 (2014), 465--500.

[35]

Selmer Bringsjord and Joshua Taylor. 2012. Introducing divine-command robot ethics. In Robot Ethics: The Ethical and Social Implication of Robotics. MIT Press, Cambridge, Massachusetts, MA, 85--108.

[36]

Miles Brundage. 2014. Limitations and risks of machine ethics. Journal of Experimental 8 Theoretical Artificial Intelligence 26, 3 (2014), 355--372.

[37]

José-Antonio Cervantes, Luis-Felipe Rodríguez, Sonia López, and Félix Ramos. 2013. A biologically inspired computational model of moral decision making for autonomous agents. In Proceedings of the 12th IEEE International Conference on Cognitive Informatics 8 Cognitive Computing (ICCI* CC’13). IEEE, New York, NY, 111--117.

[38]

José-Antonio Cervantes, Luis-Felipe Rodríguez, Sonia López, Félix Ramos, and Francisco Robles. 2016. Autonomous agents and ethical decision-making. Cogn. Comput. 8, 2 (2016), 278--296.

[39]

Ying Chen, J. D. Elenee Argentinis, and Griff Weber. 2016. IBM Watson: How cognitive computing can be applied to big data challenges in life sciences research. Clin. Therapeut. 38, 4 (2016), 688--701.

[40]

Christopher Cloos. 2005. The Utilibot project: An autonomous mobile robot based on utilitarianism. In Machine Ethics: Papers from the 2005 AAAI Fall Symposium. AAAI Press, Menlo Park, CA, 38--45.

[41]

Nicolas Cointe, Grégory Bonnet, and Olivier Boissier. 2016. Ethical judgment of agents’ behaviors in multi-agent systems. In Proceedings of the 2016 International Conference on Autonomous Agents 8 Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1106--1114.

[42]

Kari Gwen Coleman. 2001. Android arete: Toward a virtue ethic for computational agents. Ethics Inf. Technol. 3, 4 (2001), 247--265.

Digital Library

[43]

Benjamin Constant. 2013. Des réactions Politiques. Presses Électroniques de France, Paris,France.

[44]

David Copp. 2005. The Oxford Handbook of Ethical Theory. Oxford University Press, Oxford, UK.

[45]

Missy Cummings. 2017. Artificial Intelligence and the Future of Warfare. Technical Report. International Security Department and US and the Americas Program.

[46]

Jonathan Dancy. 1999. Can a particularist learn the difference between right and wrong? In Proceedings of the 20th World Congress of Philosophy, Vol. 1. Philosophy Documentation Center, Charlottesville, VA, 59--72.

[47]

Jonathan Dancy. 2000. The particularist’s progress. In Moral Particularism. Oxford University Press, Oxford, UK.

[48]

Jonathan Dancy et al. 2004. Ethics without Principles. Oxford University Press on Demand, Oxford, UK.

[49]

David Davenport. 2014. Moral mechanisms. Philos. Technol. 27, 1 (2014), 47--60.

[50]

Morteza Dehghani, Emmett Tomai, Kenneth D. Forbus, and Matthew Klenk. 2008. An integrated reasoning approach to moral decision-making. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’08). AAAI Press, Palo Alto, CA, 1280--1286.

[51]

Louise Dennis, Michael Fisher, Marija Slavkovik, and Matt Webster. 2016. Formal verification of ethical choices in autonomous systems. Robot. Auton. Syst. 77 (2016), 1--14. https://doi.org/10.1016/j.robot.2015.11.012

Digital Library

[52]

Louise Abigail Dennis, Michael Fisher, and Alan F. T. Winfield. 2015. Towards verifiably ethical robot behaviour. In Proceedings of the AAAI Workshop: AI and Ethics. AAAI Press, Palo Alto, CA, 45--52.

[53]

Kathleen M. Eisenhardt. 1989. Building theories from case study research. Acad. Manage. Rev. 14, 4 (1989), 532--550.

[54]

Amitai Etzioni and Oren Etzioni. 2017. Incorporating ethics into artificial intelligence. J. Ethics 21, 4 (2017), 403--418.

[55]

Ralph Evins, Ravi Vaidyanathan, and Stuart Burgess. 2014. Multi-material compositional pattern-producing networks for form optimisation. In Proceedings of the European Conference on the Applications of Evolutionary Computation. Springer, Berlin, 189--200.

[56]

Daniel M. T. Fessler, H. Clark Barrett, Martin Kanovsky, Stephen Stich, Colin Holbrook, Joseph Henrich, Alexander H. Bolyanatz, Matthew M. Gervais, Michael Gurven, Geoff Kushnick, et al. 2015. Moral parochialism and contextual contingency across seven societies. Proc. Roy. Soc. B: Biol. Sci. 282, 1813 (2015), 20150907.

[57]

Luciano Floridi and Jeff W. Sanders. 2004. On the morality of artificial agents. Minds Mach. 14, 3 (2004), 349--379.

Digital Library

[58]

Harry Frankfurt. 1994. An alleged asymmetry between actions and omissions. Ethics 104, 3 (1994), 620--623.

[59]

Stan Franklin and F. G. Patterson Jr. 2006. The LIDA architecture: Adding new modes of learning to an intelligent autonomous, software agent. In Proceedings of 9th World Conference on Integrated Design 8 Process Technology (IDPT'06) 703, 764--1004. https://www.researchgate.net/publication/210304626_The_LIDA_architecture_Adding_new_modes_of_learning_to_an_intelligent_autonomous_software_agent.

[60]

Ulrich Furbach, Claudia Schon, and Frieder Stolzenburg. 2014. Automated reasoning in deontic logic. In Proceedings of the International Workshop on Multi-disciplinary Trends in Artificial Intelligence. Springer, Cham, Switzerland, 57--68.

Digital Library

[61]

Iason Gabriel. 2020. Artificial intelligence, values and alignment. ArXiv:2001.09768. Retrieved from https://arxiv.org/abs/2001.09768.

[62]

Jean-Gabriel Ganascia. 2007. Ethical system formalization using non-monotonic logics. In Proceedings of the Cognitive Science Conference, Vol. 29. Cognitive Science Society, Nashville, TN, 1013--1018.

[63]

Jean-Gabriel Ganascia. 2012. An agent-based formalization for resolving ethical conflicts. In Proceedings of the ECAI Belief Change, Non-monotonic Reasoning and Conflict Resolution Workshop). IOD Press, Amsterdam, The Netherlands.

[64]

Bryce Goodman and Seth Flaxman. 2017. European union regulations on algorithmic decision making and a “right to explanation.”AI Mag. 38, 3 (2017), 50--57.

[65]

Naveen Sundar Govindarajulu and Selmer Bringsjord. 2017. On automating the doctrine of double effect. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17). 4722--4730.

[66]

Marcello Guarini. 2006. Particularism and the classification and reclassification of moral cases. IEEE Intell. Syst. 21, 4 (2006), 22--28.

Digital Library

[67]

M. Guarini. 2012. Moral cases, moral reasons, and simulation. AISB/IACAP World Congr. 21, 4 (2012), 22--28.

[68]

Claire Gudex, Paul Kind, et al. 1988. The QALY Toolkit. Technical Report. Centre for Health Economics, University of York.

[69]

Joram Graf Haber. 1993. Doing and Being, Selected Readings in Moral Philosophy. Vol. 208. Macmillan, London, UK.

[70]

Carole D. Hafner and Donald H. Berman. 2002. The role of context in case-based legal reasoning: Teleological, temporal, and procedural. Artif. Intell. Law 10, 1--3 (2002), 19--64.

Digital Library

[71]

Joseph Y. Halpern and Moshe Y. Vardi. 1991. Model checking vs. theorem proving: a manifesto. In Proceedings of the Second International Conference on Principles of Knowledge Representation and Reasoning (KR'91). 325--334. https://dl.acm.org/doi/10.5555/3087158.3087191

[72]

The Han, Ari Saptawijaya, and Luís Moniz Pereira. 2012. Moral reasoning under uncertainty. In Logic for Programming, Artificial Intelligence, and Reasoning. Springer, Berlin, 212--227.

[73]

Richard Mervyn Hare and R. M. Hare. 1963. Freedom and Reason. Vol. 92. Oxford Paperbacks, Oxford, UK.

[74]

Gilbert Harman. 2005. Moral particularism and transduction. Philos. Issues 15 (2005), 44--55.

[75]

Gilbert Harman and Judith Jarvis Thomson. 1996. Moral Relativism. Blackwell Publishers, Cambridge, MA, 3--64.

[76]

Patrick Chisan Hew. 2014. Artificial moral agents are infeasible with foreseeable technologies. Ethics Inf. Technol. 16, 3 (2014), 197--206.

Digital Library

[77]

Kenneth Einar Himma. 2009. Artificial agency, consciousness, and the criteria for moral agency: What properties must an artificial agent have to be a moral agent? Ethics Inf. Technol. 11, 1 (2009), 19--29.

Digital Library

[78]

Ali Reza Honarvar and Nasser Ghasem-Aghaee. 2009. An artificial neural network approach for creating an ethical artificial agent. In Proceedings of the IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA’09). IEEE, Los Alamitos, CA, 290--295.

[79]

Ali Reza Honarvar and Nasser Ghasem-Aghaee. 2009. Casuist BDI-agent: A new extended BDI architecture with the capability of ethical reasoning. In Proceedings of the International Conference on Artificial Intelligence and Computational Intelligence. Springer, Berlin, 86--95.

Digital Library

[80]

John F. Horty. 2001. Agency and Deontic Logic. Oxford University Press, Oxford, UK.

[81]

John F. Horty and Nuel Belnap. 1995. The deliberative stit: A study of action, omission, ability, and obligation. J. Philos. Logic 24, 6 (1995), 583--644.

[82]

Don Howard and Ioan Muntean. 2017. Artificial moral cognition: Moral functionalism and autonomous moral agency. In Philosophy and Computing. Springer, Cham, Switzerland, 121--159.

[83]

Matthew Iklé, Arthur Franz, Rafal Rzepka, and Ben Goertzel. 2018. Proceedings of the 11th International Conference on Artificial General Intelligence (AGI’18). Vol. 10999. Springer, Heidelberg, Germany.

[84]

Kant Immanuel. 1785. Groundwork of the metaphysics of morals. In Late Modern Philosophy: Essential Readings with Commentary, Elizabeth Schmidt Radcliffe, Richard McCarty, Fritz Allhoff, and Anand Vaidya (Eds.). Blackwell, Hoboken, NJ.

[85]

Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. 1998. Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 1--2 (1998), 99--134.

Digital Library

[86]

Frances Myrna Kamm et al. 2007. Intricate Ethics: Rights, Responsibilities, and Permissible Harm. Oxford University Press, Oxford, UK.

[87]

Solomon Kullback and Richard A. Leibler. 1951. On Information and Sufficiency. Ann. Math. Stat. 22, 1 (1951), 79--86.

[88]

Jamy Li, Xuan Zhao, Mu-Jung Cho, Wendy Ju, and Bertram F. Malle. 2016. From Trolley to Autonomous Vehicle: Perceptions of Responsibility and Moral Norms in Traffic Accidents with Self-driving Cars. Technical Report. SAE Technical Paper.

[89]

Felix Lindner, Martin Mose Bentzen, and Bernhard Nebel. 2017. The HERA approach to morally competent robots. In IROS 2017: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, Los Alamitos, CA, 6991--6997.

[90]

Edouard Machery. 2017. Philosophy within Its Proper Bounds. Oxford University Press, Oxford, UK, Chapter 2, 45--89.

[91]

John L. Mackie. 1977. Ethics: Inventing Right and Wrong. Penguin Books, Frankfurt, Germany.

[92]

Tamas Madl and Stan Franklin. 2015. Constrained incrementalist moral decision making for a biologically inspired cognitive architecture. In A Construction Manual for Robots’ Ethical Systems. Springer, Cham, Switzerland, 137--153.

[93]

Bertram F. Malle, Matthias Scheutz, and Joseph L. Austerweil. 2017. Networks of social and moral norms in human and robot agents. In A World with Robots. Springer, Cham, Switzerland, 3--17.

[94]

Bruce M. McLaren. 2003. Extensionally defining principles and cases in ethics: An AI model. Artif. Intell. 150, 1--2 (2003), 145--181.

Digital Library

[95]

Bruno Mermet and Gaële Simon. 2016. Formal verication of ethical properties in multiagent systems. In Proceedings of the 1st Workshop on Ethics in the Design of Intelligent Agents. CEUR, The Hague, Netherlands, 26--31.

[96]

John Mikhail. 2007. Universal moral grammar: Theory, evidence and the future. Trends Cogn. Sci. 11, 4 (2007), 143--152.

[97]

Alex Miller. 2003. An Introduction to Contemporary Metaethics. Polity, Cambridge, UK.

[98]

James H. Moor. 2006. The nature, importance, and difficulty of machine ethics. IEEE Intell. Syst. 21, 4 (2006), 18--21.

Digital Library

[99]

Yuko Murakami. 2005. Utilitarian deontic logic. Adv. Modal Logic 5 (2005), 211--230.

[100]

Baldoino F. dos S. Neto, Viviane Torres da Silva, and Carlos José Pereira de Lucena. 2011. NBDI: An architecture for goal-oriented normative agents. In Proceedings of the 3rd International Conference on Agents and Artificial Intelligence (ICAART’11), Vol. 1. Springer, Berlin, 116--125.

[101]

Ritesh Noothigattu, Snehalkumar “Neil” S. Gaikwad, Edmond Awad, Sohan Dsouza, Iyad Rahwan, Pradeep Ravikumar, and Ariel D. Procaccia. 2018. A voting-based system for ethical decision making. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence 2018. AAAI Press, Palo Alto, CA, 1587--1594.

[102]

Luís Moniz Pereira and Ari Saptawijaya. 2007. Modelling morality with prospective logic. In Proceedings of the Portuguese Conference on Artificial Intelligence. Springer, Berlin, 99--111.

[103]

Luís Moniz Pereira and Ari Saptawijaya. 2011. Modelling Morality with Prospective Logic. Cambridge University Press, Cambridge, UK, 398--421.

[104]

Luís Moniz Pereira and Ari Saptawijaya. 2016. Programming Machine Ethics. Vol. 26. Springer, Cham, Switzerland.

[105]

Luís Moniz Pereira and Ari Saptawijaya. 2017. Counterfactuals, logic programming and agent morality. In Applications of Formal Philosophy. Springer, Berlin, 25--53.

[106]

Rosalind Wright Picard et al. 1995. Affective Computing. MIT Press, Cambridge, MA.

[107]

Javier Pinto and Raymond Reiter. 1993. Temporal reasoning in logic programming: A case for the situation calculus. In Proceedings of the 10th International Conference in Logic Programming, Vol. 93. Springer, Berlin, 203--221.

[108]

Matthijs Pontier and Johan Hoorn. 2012. Toward machines that behave ethically better than humans do. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 34. Cognitive Science Society, Austin, TX, 2198--2203.

[109]

Matthijs A. Pontier and Guy A. M. Widdershoven. 2013. Robots that stimulate autonomy. In Artificial Intelligence Applications and Innovations, Harris Papadopoulos, Andreas S. Andreou, Lazaros Iliadis, and Ilias Maglogiannis (Eds.). Springer, Berlin, 195--204.

[110]

Jesse Prinz. 2007. The Emotional Construction of Morals. Oxford University Press, Oxford, UK.

[111]

James Rachels. 1979. Active and Passive Euthanasia. Springer, Boston, MA, 551--516.

[112]

John Rawls. 2009. A Theory of Justice. Harvard University Press, London, UK.

[113]

Gregory S. Reed, Mikel D. Petty, Nicholaos J. Jones, Anthony W. Morris, John P. Ballenger, and Harry S. Delugach. 2016. A principles-based model of ethical considerations in military decision making. J. Defense Model. Simul. 13, 2 (2016), 195--211.

[114]

Daniel Richards and Martyn Amos. 2014. Evolving morphologies with CPPN-NEAT and a dynamic substrate. In Proceedings of the Artificial Life Conference Proceedings 14. MIT Press, Cambridge, MA, 255--262.

[115]

Stuart J. Russell and Peter Norvig. 2016. Artificial Intelligence: A Modern Approach. Pearson Education Limited, Kuala Lumpur, Malaysia.

[116]

Rafal Rzepka and Kenji Araki. 2017. What people say? Web-based casuistry for artificial morality experiments. In Proceedings of the International Conference on Artificial General Intelligence. Springer, Cham, Switzerland, 178--187.

[117]

Lydia Saad. 2010. Four Moral Issues Sharply Divide Americans. Retrieved December 6, 2019 from https://news.gallup.com/poll/137357/four-moral-issues-sharply-divide-americans.aspx.

[118]

Sonya Sachdeva, Purnima Singh, and Douglas Medin. 2011. Culture and the quest for universal principles in moral reasoning. Int. J. Psychol. 46, 3 (2011), 161--176.

[119]

Filippo Santoni de Sio and Jeroen Van den Hoven. 2018. Meaningful human control over autonomous systems: A philosophical account. Front. Robot. AI 5 (2018), 15. https://doi.org/10.3389/frobt.2018.00015

[120]

Ari Saptawijaya and Luís Moniz Pereira. 2014. Towards modeling morality computationally with logic programming. In Proceedings of the International Symposium on Practical Aspects of Declarative Languages. Springer, Berlin, Heidelberg, 104--119.

Digital Library

[121]

Ari Saptawijaya and Luís Moniz Pereira. 2015. The potential of logic programming as a computational tool to model morality. In A Construction Manual for Robots’ Ethical Systems. Springer, Berlin, 169--210.

[122]

Ari Saptawijaya, Luís Moniz Pereira, et al. 2012. Moral reasoning under uncertainty. In Proceedings of the International Conference on Logic for Programming Artificial Intelligence and Reasoning. Springer, Berlin, 212--227.

[123]

Matthias Scheutz and Bertram F. Malle. 2014. Think and do the right thing: A plea for morally competent autonomous robots. In Proceedings of the IEEE 2014 International Symposium on Ethics in Engineering, Science, and Technology. IEEE Press, Los Alamitos, CA, 9.

[124]

Eric Schwitzgebel and Fiery Cushman. 2012. Expertise in moral reasoning? Order effects on moral judgment in professional philosophers and non-philosophers. Mind Lang. 27, 2 (2012), 135--153.

[125]

Eric Schwitzgebel and Fiery Cushman. 2015. Philosophers’ biased judgments persist despite training, expertise and reflection. Cognition 141 (2015), 127--137. https://doi.org/10.1016/j.cognition.2015.04.015

[126]

Jaeeun Shim, Ronald Arkin, and Michael Pettinatti. 2017. An intervening ethical governor for a robot mediator in patient-caregiver relationship: Implementation and evaluation. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA’17). IEEE, Los Alamitos, CA, 2936--2942.

[127]

Robert Sparrow. 2007. Killer robots. J. Appl. Philos. 24, 1 (2007), 62--77.

[128]

Gopal Sreenivasan. 2002. Errors about errors: Virtue theory and trait attribution. Mind 111, 441 (2002), 47--68.

[129]

Kenneth O. Stanley, David B. D’Ambrosio, and Jason Gauci. 2009. A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15, 2 (2009), 185--212.

Digital Library

[130]

Kenneth O. Stanley and Risto Miikkulainen. 2002. Evolving neural networks through augmenting topologies. Evol. Comput. 10, 2 (2002), 99--127.

Digital Library

[131]

John P. Sullins. 2006. When is a robot a moral agent? Int. Rev. Inf. Ethics 6, 12 (2006), 23--30. https://philpapers.org/rec/SULWIA-2.

[132]

Sarah M. Thornton, Selina Pan, Stephen M. Erlien, and J. Christian Gerdes. 2017. Incorporating ethical considerations into automated vehicle control. IEEE Trans. Intell. Transport. Syst. 18, 6 (2017), 1429--1439.

Digital Library

[133]

Mihnea Tufiş and Jean-Gabriel Ganascia. 2015. Grafting norms onto the BDI agent model. In A Construction Manual for Robots’ Ethical Systems. Springer, Cham, Switzerland, 119--133.

[134]

Matteo Turilli. 2007. Ethical protocols design. Ethics Inf. Technol. 9, 1 (2007), 49--62.

Digital Library

[135]

Chien Van Dang, Tin Trung Tran, Ki-Jong Gil, Yong-Bin Shin, Jae-Won Choi, Geon-Soo Park, and Jong-Wook Kim. 2017. Application of soar cognitive agent based on utilitarian ethics theory for home service robots. In Proceedings of the 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI’17). IEEE, Los Alamitos, CA, 155--158.

[136]

Marlies Van de Voort, Wolter Pieters, and Luca Consoli. 2015. Refining the ethics of computer-made decisions: A classification of moral mediation by ubiquitous machines. Ethics Inf. Technol. 17, 1 (2015), 41--56.

Digital Library

[137]

Dieter Vanderelst and Alan Winfield. 2017. An architecture for ethical robots inspired by the simulation theory of cognition. Cogn. Syst. Res. 48 (2017), 56--66. https://doi.org/10.1016/j.cogsys.2017.04.002

[138]

J. David Velleman. 2013. Foundations for Moral Relativism. OpenBook Publishers, Cambridge, UK.

[139]

Bart Verheij. 2016. Formalizing value-guided argumentation for ethical systems design. Artif. Intell. Law 24, 4 (2016), 387--407.

Digital Library

[140]

John Voiklis, Boyoung Kim, Corey Cusimano, and Bertram F. Malle. 2016. Moral judgments of human vs. robot agents. In Proceedings of the 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’16). IEEE, Los Alamitos, CA, 775--780.

[141]

Wendell Wallach. 2010. Cognitive models of moral decision making. Topics Cogn. Sci. 2, 3 (2010), 420--429.

[142]

Wendell Wallach and Colin Allen. 2008. Moral Machines: Teaching Robots Right from Wrong. Oxford University Press, Oxford, UK.

Digital Library

[143]

Wendell Wallach, Colin Allen, and Iva Smit. 2008. Machine morality: Bottom-up and top-down approaches for modelling human moral faculties. Ai Soc. 22, 4 (2008), 565--582.

[144]

Wendell Wallach, Stan Franklin, and Colin Allen. 2010. A conceptual and computational model of moral decision making in human and artificial agents. Top. Cogn. Sci. 2, 3 (2010), 454--485.

[145]

Richard Whitley. 2000. The Intellectual and Social Publisher of the Sciences. Oxford University Press on Demand, Oxford, UK.

[146]

Vincent Wiegel and Jan van den Berg. 2009. Combining moral theory, modal logic and MAS to create well-behaving artificial agents. Int. J. Soc. Robot. 1, 3 (2009), 233--242.

[147]

Alan F. T. Winfield, Christian Blum, and Wenguo Liu. 2014. Towards an ethical robot: Internal models, consequences and ethical action selection. In Proceedings of the Conference Towards Autonomous Robotic Systems. Springer, Cham, Switzerland, 85--96.

[148]

Michael Wooldridge and Wiebe Van Der Hoek. 2005. On obligations and normative ability: Towards a logical analysis of the social contract. J. Appl. Logic 3, 3--4 (2005), 396--420.

[149]

Fiona Woollard. 2012. The doctrine of doing and allowing II: The moral relevance of the doing/allowing distinction. Philos. Compass 7, 7 (2012), 459--469.

[150]

Fiona Woollard. 2015. Doing and Allowing Harm. Oxford University Press, Oxford, UK.

[151]

Yueh-Hua Wu and Shou-De Lin. 2017. A low-cost ethics shaping approach for designing reinforcement learning agents. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI-18). AAAI Press, Palo Alto, CA, 1687--1694.

[152]

Levent Yilmaz, Ana Franco-Watkins, and Timothy S. Kroecker. 2017. Computational models of ethical decision-making: A coherence-driven reflective equilibrium model. Cogn. Syst. Res. 46 (2017), 61--74. https://doi.org/10.1016/j.cogsys.2017.02.005

Digital Library

[153]

H. Yu, Z. Shen, C. Leung C. Miao, V. R. Lesser, and Q. Yang. 2018. Building ethics into artificial intelligence. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). 5527--5533.

[154]

Eliezer Yudkowsky. 2001. Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures. The Singularity Institute, San Francisco, USA.

[155]

Tal Zarsky. 2016. The trouble with algorithmic decisions: An analytic road map to examine efficiency and fairness in automated and opaque decision making. Sci. Technol. Hum. Values 41, 1 (2016), 118--132.

Cited By

Wang YTan GSi H(2025)Decision modeling for automated driving in dilemmas based on bidirectional value alignment of moral theory values and fair human moral valuesTransportation Research Part F: Traffic Psychology and Behaviour10.1016/j.trf.2024.11.001108(14-27)Online publication date: Jan-2025
https://doi.org/10.1016/j.trf.2024.11.001
Greenberg A(2025)Ethics for artificial agentsInterdependent Human-Machine Teams10.1016/B978-0-443-29246-0.00001-8(349-371)Online publication date: 2025
https://doi.org/10.1016/B978-0-443-29246-0.00001-8
Lamberti PBombaerts GIJsselsteijn W(2025)Mind the gap: bridging the divide between computer scientists and ethicists in shaping moral machinesEthics and Information Technology10.1007/s10676-024-09806-127:1Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s10676-024-09806-1
Show More Cited By

Index Terms

Implementations in Machine Ethics: A Survey
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence
2. Social and professional topics

Recommendations

Conative Dimensions of Machine Ethics: A Defense of Duty

Immanuel Kant is one of the giants of moral theorizing in the western philosophical tradition. He developed a view of moral imperatives and duty that continues to inspire thought up to the present. In a thought-provoking series of papers, Anthony ...
Robots: ethical by design

Among ethicists and engineers within robotics there is an ongoing discussion as to whether ethical robots are possible or even desirable. We answer both of these questions in the positive, based on an extensive literature study of existing arguments. ...
A Challenge for Machine Ethics

That the successful development of fully autonomous artificial moral agents (AMAs) is imminent is becoming the received view within artificial intelligence research and robotics. The discipline of Machines Ethics, whose mandate is to create such ethical ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 53, Issue 6

Invited Tutorial and Regular Papers

November 2021

803 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3441629

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 December 2020

Accepted: 01 August 2020

Revised: 01 April 2020

Received: 01 May 2019

Published in CSUR Volume 53, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

armasuisse Science and Technology (S+T), via the Swiss Center for Drones and Robotics of the Department of Defense, Civil Protection and Sport (DDPS)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

81
Total Citations
View Citations
2,444
Total Downloads

Downloads (Last 12 months)369
Downloads (Last 6 weeks)28

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YTan GSi H(2025)Decision modeling for automated driving in dilemmas based on bidirectional value alignment of moral theory values and fair human moral valuesTransportation Research Part F: Traffic Psychology and Behaviour10.1016/j.trf.2024.11.001108(14-27)Online publication date: Jan-2025
https://doi.org/10.1016/j.trf.2024.11.001
Greenberg A(2025)Ethics for artificial agentsInterdependent Human-Machine Teams10.1016/B978-0-443-29246-0.00001-8(349-371)Online publication date: 2025
https://doi.org/10.1016/B978-0-443-29246-0.00001-8
Lamberti PBombaerts GIJsselsteijn W(2025)Mind the gap: bridging the divide between computer scientists and ethicists in shaping moral machinesEthics and Information Technology10.1007/s10676-024-09806-127:1Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s10676-024-09806-1
Cheong B(2024)The Rise of AI Avatars: Legal Personhood, Rights and Liabilities in an Evolving MetaverseJournal of Digital Technologies and Law10.21202/jdtl.2024.422:4(857-885)Online publication date: 28-Dec-2024
https://doi.org/10.21202/jdtl.2024.42
Memon MAutili MFilippone GScoccia GInverardi PFilkov VRay BZhou M(2024)A High-level Architecture of an Automated Context-aware Ethics-based Negotiation ApproachProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695332(2468-2469)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695332
Chivukula SGray CLi ZPivonka AChen J(2024)Surveying a Landscape of Ethics-Focused Design MethodsACM Journal on Responsible Computing10.1145/36789881:3(1-32)Online publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1145/3678988
Woodgate JAjmeri N(2024)Macro Ethics Principles for Responsible AI Systems: Taxonomy and DirectionsACM Computing Surveys10.1145/367239456:11(1-37)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3672394
Angelopoulos GEstivill-Castro VGrollman DBroadbent EJu WSoh HWilliams T(2024)Human-Robot Dialogue that Elicits The Alignment of Moral Principles For Driverless VehiclesCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640641(196-200)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640641
Peterson CBroersen J(2024)Understanding the Limits of Explainable Ethical AIInternational Journal on Artificial Intelligence Tools10.1142/S021821302460001733:03Online publication date: 9-Jan-2024
https://doi.org/10.1142/S0218213024600017
Kolker SDennis LPereira RXu M(2024)Applying Ethical Decision Making on Space Missions2024 IEEE Space Computing Conference (SCC)10.1109/SCC61854.2024.00022(136-138)Online publication date: 15-Jul-2024
https://doi.org/10.1109/SCC61854.2024.00022
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents