What Should a Robot Do? Comparing Human and Large Language Model Recommendations for Robot Deception
Authors: 
Kantwon Rogers, Reiden John Allen Webber, Geronimo Gorostiaga Zubizarreta, Arthur Melo Cruz, Shengkang Chen, Ronald C. Arkin, Jason Borenstein, Alan R. WagnerAuthors Info & Claims
Kantwon Rogers, Reiden John Allen Webber, Geronimo Gorostiaga Zubizarreta, Arthur Melo Cruz, Shengkang Chen, Ronald C. Arkin, Jason Borenstein, Alan R. WagnerAuthors Info & ClaimsPages 906 - 910
Abstract
This study compares human ethical judgments with Large Language Models (LLMs) on robotic deception in various scenarios. Surveying human participants and querying LLMs, we presented ethical dilemmas in high-risk and low-risk contexts. Findings reveal alignment between humans and LLMs in high-risk scenarios, prioritizing safety, but notable divergences in low-risk situations, reflecting challenges in AI development to accurately capture human social nuances and moral expectations.
Supplemental Material
MP4 File
Supplemental video
- Download
- 26.47 MB
References
[1]
Marwa Abdulhai, Gregory Serapio-Garcia, Clément Crepy, Daria Valter, John Canny, and Natasha Jaques. 2023. Moral foundations of large language models. arXiv preprint arXiv:2310.15337 (2023).
[2]
Jason Borenstein, Arthur Melo Cruz, Alan Wagner, and Ronald Arkin. 2023. From HHI to HRI: Which Facets of Ethical Decision-Making Should Inform a Robot?. In International Conference on Computer Ethics, Vol. 1.
[3]
Shengkang Chen, Vidullan Surendran, Alan R Wagner, Jason Borenstein, and Ronald C Arkin. 2022. Toward Ethical Robotic Behavior in Human-Robot Interaction Scenarios. arXiv preprint arXiv:2206.10727 (2022).
[4]
Mihaela Constantinescu, Constantin Vica, Radu Uszkai, and Cristina Voinea. 2022. Blame it on the AI? On the moral responsibility of artificial moral advisors. Philosophy & Technology 35, 2 (2022), 35.
[5]
D Ettinger and P Jehiel. 2009. Towards a theory of deception: ELSE Working Papers (181). ESRC Centre for Economic Learning and Social Evolution, London, UK (2009).
[6]
Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, and William Saunders. 2021. Truthful AI: Developing and governing AI that does not lie. arXiv preprint arXiv:2110.06674 (2021).
[7]
Stephen Fitz. 2023. Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddings. arXiv preprint arXiv:2309.09397 (2023).
[8]
Andrew Hundt, William Agnew, Vicky Zeng, Severin Kacianka, and Matthew Gombolay. 2022. Robots enact malignant stereotypes. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 743--756.
[9]
AM Isaac and Will Bridewell. 2017. Why robots need to deceive (and how). Robot ethics 2 (2017), 157--172.
[10]
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1--38.
[11]
Markus Kneer. 2020. Can a robot lie? (2020).
[12]
Sebastian Krügel, Andreas Ostermaier, and Matthias Uhl. 2022. Zombies in the loop? Humans trust untrustworthy AI-advisors for ethical decisions. Philosophy & Technology 35, 1 (2022), 17.
[13]
Sebastian Krügel, Andreas Ostermaier, and Matthias Uhl. 2023. The moral authority of ChatGPT. arXiv preprint arXiv:2301.07098 (2023).
[14]
Florian Leiser, Sven Eckhardt, Merlin Knaeble, Alexander Maedche, Gerhard Schwabe, and Ali Sunyaev. 2023. From ChatGPT to FactGPT: A Participatory Design Study to Mitigate the Effects of Large Language Model Hallucinations on Users. In Proceedings of Mensch und Computer 2023. 81--90.
[15]
Yaniv Leviathan and Yossi Matias. 2018. Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone. https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html
[16]
Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning for negotiation dialogues. arXiv preprint arXiv:1706.05125 (2017).
[17]
Bertram Malle. 2019. How many dimensions of mind perception really are there?. In CogSci. 2268--2274.
[18]
Stuart McLennan, Amelia Fiske, Daniel Tigard, Ruth Müller, Sami Haddadin, and Alena Buyx. 2022. Embedded ethics: a proposal for integrating ethics into the development of medical AI. BMC Medical Ethics 23, 1 (2022), 6.
[19]
OpenAI. 2023. GPT-4 Technical Report. https://doi.org/10.48550/ARXIV.2303.08774
[20]
Aida Ramezani and Yang Xu. 2023. Knowledge of cultural moral norms in large language models. arXiv preprint arXiv:2306.01857 (2023).
[21]
Stefan Sarkadi. 2021. Deception. Ph. D. Dissertation.
[22]
Amanda Sharkey and Noel Sharkey. 2021. We need to talk about deception in social robotics! Ethics and Information Technology 23 (2021), 309--316.
[23]
Jaeeun Shim and Ronald C Arkin. 2013. A taxonomy of robot deception and its benefits in HRI. In 2013 IEEE international conference on systems, man, and cybernetics. IEEE, 2328--2335.
[24]
Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, and Monojit Choudhury. 2023. Exploring Large Language Models' Cognitive Moral Development through Defining Issues Test. arXiv preprint arXiv:2309.13356 (2023).
[25]
Alan R Wagner and Ronald C Arkin. 2009. Robot deception: recognizing when a robot should deceive. In 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation-(CIRA). IEEE, 46--54.
[26]
Kara Weisman, Carol S Dweck, and Ellen M Markman. 2017. Rethinking people's conceptions of mental life. Proceedings of the National Academy of Sciences 114, 43 (2017), 11374--11379.
[27]
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, and Fei Xia. 2023. Language to Rewards for Robotic Skill Synthesis. https://doi.org/10.48550/ARXIV.2306.08647
[28]
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv preprint arXiv:2309.01219 (2023).
Index Terms
- What Should a Robot Do? Comparing Human and Large Language Model Recommendations for Robot Deception
Recommendations
Robot Betrayal: a guide to the ethics of robotic deception
AbstractIf a robot sends a deceptive signal to a human user, is this always and everywhere an unethical act, or might it sometimes be ethically desirable? Building upon previous work in robot ethics, this article tries to clarify and refine our ...
Integrating robot ethics and machine morality: the study and design of moral competence in robots
Robot ethics encompasses ethical questions about how humans should design, deploy, and treat robots; machine morality encompasses questions about what moral capacities a robot should have and how these capacities could be computationally implemented. ...
Perspectives on Moral Agency in Human-Robot Interaction
HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot InteractionEstablishing when, how, and why robots should be considered moral agents is key for advancing human-robot interaction. For instance, whether a robot is considered a moral agent has significant implications for how researchers, designers, and users can, ...
Comments
Information & Contributors
Information
Published In
March 2024
1408 pages
ISBN:9798400703232
DOI:10.1145/3610978
- General Chairs:
- Dan Grollman,
- Elizabeth Broadbent,
- Program Chairs:
- Wendy Ju,
- Harold Soh,
- Tom Williams
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 March 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
Conference
HRI '24: ACM/IEEE International Conference on Human-Robot Interaction
March 11 - 15, 2024
CO, Boulder, USA
Acceptance Rates
Overall Acceptance Rate 268 of 1,124 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 264Total Downloads
- Downloads (Last 12 months)264
- Downloads (Last 6 weeks)39
Reflects downloads up to 26 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in