research-article

Punishable AI: Examining Users' Attitude Towards Robot Punishment

Authors:

Sarah Theres Völkel,

Elias Naphausen,

Alexander Wiethoff,

Andreas MuxelAuthors Info & Claims

DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems Conference

Pages 179 - 191

https://doi.org/10.1145/3357236.3395542

Published: 03 July 2020 Publication History

Abstract

To give robots, which are black box systems for most users, feedback we have to implement interaction paradigms that users understand and accept, for example reward and punishment. In this paper we present the first HRI experience prototype which implements gradual destructive interaction, namely breaking a robot's leg as a punishment technique. We conducted an exploratory experiment (N=20) to investigate participants' behavior during the execution of three punishment techniques. Using a structured analysis of videos and interviews, we provide in-depth insights into participants' attitude towards these techniques. Participants preferred more abstract techniques and felt uncomfortable during human-like punishment interaction. Based on our findings, we raise questions how human-like technologies should be designed. A video documentation of the project can be found here: https://vimeo.com/348646727

References

[1]

Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4 (2014), 105--120.

Digital Library

[2]

Anja Austermann and Seiji Yamada. 2008. "Good robot", "bad robot" -- Analyzing users' feedback in a human-robot teaching task. In RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, New York, NY, USA, 41--46.

[3]

James Auger. 2014. Living with Robots: A Speculative Design Approach. Journal of Human-Robot Interaction 3, Article 1 (Feb. 2014), 23 pages.

Digital Library

[4]

Christoph Bartneck and Jun Hu. 2008. Exploring the abuse of robots. Interaction Studies 9, 3 (2008), 415--433.

[5]

Christoph Bartneck, Takayuki Kanda Kanda, Omar Mubin, and Abdullah Al Mahmud. 2007. The perception of animacy and intelligence based on a robot's embodiment. In 2007 7th IEEE-RAS International Conference on Humanoid Robots. IEEE, New York, NY, USA, 300--305.

[6]

Christoph Bartneck, Dana Kuli´ c, Elizabeth Croft, and Susana Zoghbi. 2009. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics 1, 1 (2009), 71--81.

[7]

Christoph Bartneck, Juliane Reichenbach, and Julie Carpenter. 2006. Use of Praise and Punishment in Human-Robot Collaborative Teams. In RO-MAN 2006 The 15th IEEE International Symposium on Robot and Human Interactive Communication, K. Dautenhahn (Ed.). IEEE, New York, NY, USA, 177--182.

[8]

Christoph Bartneck, Juliane Reichenbach, and Julie Carpenter. 2008. The carrot and the stick: The role of praise and punishment in human--robot interaction. Interaction Studies 9, 2 (2008), 179--203.

[9]

Christoph Bartneck, Chioke Rosalia, Rutger Menges, and Inèz Deckers. 2005. Robot abuse -- a limitation of the media equation. Proceedings of the Interact 2005 Workshop on Agent Abuse, Rome. (2005).

[10]

Christoph Bartneck, Michel Van Der Hoek, Omar Mubin, and Abdullah Al Mahmud. 2007a. "Daisy, daisy, give me your answer do!" -- Switching off a robot. In 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, New York, NY, USA, 217--222.

Digital Library

[11]

Christoph Bartneck, Marcel Verbunt, Omar Mubin, and Abdullah Al Mahmud. 2007b. To kill a mockingbird robot. In Proceedings of the ACM/IEEE international conference on Human-robot interaction. ACM, New York, NY, USA, 81--87.

Digital Library

[12]

Cynthia Breazeal. 2001. Affective interaction between humans and robots. In European Conference on Artificial Life. Springer, Berlin, Heidelberg, Germany, 582--591.

[13]

Cynthia Breazeal. 2002. Regulation and entrainment in human-robot interaction. The International Journal of Robotics Research 21, 10--11 (2002), 883--902.

Digital Library

[14]

Cynthia Breazeal. 2003. Toward sociable robots. Robotics and autonomous systems 42, 3--4 (2003), 167--175.

[15]

Cynthia Breazeal. 2004. Social interactions in HRI: the robot view. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 34, 2 (May 2004), 181--186.

Digital Library

[16]

Drazen Brsci´ c, Hiroyuki Kidokoro, Yoshitaka Suehiro, and Takayuki Kanda. 2015. Escaping from children's abuse of social robots. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI '15). ACM, New York, NY, USA, 59--66.

Digital Library

[17]

Maya Cakmak, Crystal Chao, and Andrea L. Thomaz. 2010. Designing Interactions for Robot Active Learners. IEEE Transactions on Autonomous Mental Development 2, 2 (June 2010), 108--118.

Digital Library

[18]

John S. Carton. 1996. The differential effects of tangible rewards and praise on intrinsic motivation: A comparison of cognitive evaluation theory and operant theory. The Behavior Analyst 19, 2 (1996), 237--255.

[19]

John P. Charlton. 2009. The determinants and expression of computer-related anger.computers in Human Behavior 25, 6 (2009), 1213--1221.

Digital Library

[20]

Hyojin Chin and Mun Yong Yi. 2019. Should an Agent Be Ignoring It?: A Study of Verbal Abuse Types and Conversational Agents' Response Styles. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, New York, NY, USA, Article LBW2422, 6 pages.

Digital Library

[21]

Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37--46.

[22]

Francisco Cruz, Sven Magg, Cornelius Weber, and Stefan Wermter. 2016. Training agents with interactive reinforcement learning and contextual affordances. IEEE Transactions on Cognitive and Developmental Systems 8, 4 (2016), 271--284.

[23]

Christian Daniel, Malte Viering, Jan Metz, Oliver Kroemer, and Jan Peters. 2014. Active Reward Learning. In Proceedings of Robotics: Science and Systems X. Berkeley, USA.

[24]

John J. Dudley and Per Ola Kristensson. 2018. A Review of User Interface Design for Interactive Machine Learning. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 2 (2018), 1--37.

Digital Library

[25]

Malin Eiband, Mohamed Khamis, Emanuel von Zezschwitz, Heinrich Hussmann, and Florian Alt. 2017. Understanding Shoulder Surfing in the Wild: Stories from Users and Observers. In Proceedings of the 35th Annual ACM Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4254 -- 4265.

Digital Library

[26]

Julia Fink. 2012. Anthropomorphism and human likeness in the design of robots and human-robot interaction. In International Conference on Social Robotics. Springer, Berlin, Heidelberg, Germany, 199--208.

Digital Library

[27]

Michael A. Goodrich, Alan C. Schultz, and others. 2008. Human--robot interaction: a survey. Foundations and Trends® in Human--Computer Interaction 1, 3 (2008), 203--275.

Digital Library

[28]

Matthias Hoenen, Katrin T. Lübke, and Bettina M. Pause. 2016. Non-anthropomorphic robots as social entities on a neurophysiological level.computers in Human Behavior 57 (2016), 182 -- 186.

Digital Library

[29]

Aike C. Horstmann, Nikolai Bock, Eva Linhuber, Jessica M. Szczuka, Carolin Straßmann, and Nicole C Krämer. 2018. Do a robot's social skills and its objection discourage interactants from switching the robot off? PloS one 13, 7 (2018), e0201581.

[30]

Peter H. Kahn Jr, Takayuki Kanda, Hiroshi Ishiguro, Nathan G. Freier, Rachel L. Severson, Brian T. Gill, Jolina H. Ruckert, and Solace Shen. 2012. ?Robovie, you'll have to go into the closet now": Children's social and moral relationships with a humanoid robot. Developmental Psychology 48, 2 (2012), 303.

[31]

Irene Lopatovska and Harriet Williams. 2018. Personification of the Amazon Alexa: BFF or a Mindless Companion. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR '18). ACM, New York, NY, USA, 265--268.

Digital Library

[32]

Qinggang Meng, Ibrahim Tholley, and Paul WH Chung. 2014. Robots learn to dance through interaction with humans. Neural Computing and Applications 24, 1 (2014), 117--124.

[33]

Stanley Milgram. 1963. Behavioral study of obedience. The Journal of Abnormal and Social Psychology 67, 4 (1963), 371.

[34]

Clifford Nass and Youngme Moon. 2000. Machines and mindlessness: Social responses to computers. Journal of Social Issues 56, 1 (2000), 81--103.

[35]

Tatsuya Nomura, Takayuki Kanda, Hiroyoshi Kidokoro, Yoshitaka Suehiro, and Sachie Yamada. 2016. Why do children abuse robots? Interaction Studies 17, 3 (2016), 347--369.

[36]

Byron Reeves and Clifford Ivar Nass. 1996. The media equation: How people treat computers, television, and new media like real people and places. Cambridge University Press, Cambridge, UK.

Digital Library

[37]

Julia Ringler and Holger Reckter. 2012. DESU 100: about the temptation to destroy a robot. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI'12). ACM, New York, NY, USA, 151--152.

Digital Library

[38]

Chioke Rosalia, Rutger Menges, Inèz Deckers, and Christoph Bartneck. 2005. Cruelty towards robots. Robot Workshop -- Designing Robot Applications for Everyday Use, Göteborg. (2005).

[39]

Astrid M. Rosenthal-Von Der Pütten, Frank P. Schulte, Sabrina C. Eimler, Sabrina Sobieraj, Laura Hoffmann, Stefan Maderwald, Matthias Brand, and Nicole C. Krämer. 2014. Investigations on empathy towards humans and robots using fMRI.computers in Human Behavior 33 (2014), 201--212.

Digital Library

[40]

Pericle Salvini, Gaetano Ciaravella, Wonpil Yu, Gabriele Ferri, Alessandro Manzi, Barbara Mazzolai, Cecilia Laschi, Sang-Rok Oh, and Paolo Dario. 2010. How safe are service robots in urban environments? Bullying a robot. In 19th International Symposium in Robot and Human Interactive Communication. IEEE, New York, NY, USA, 1--7.

[41]

Suleman Shahid, Emiel Krahmer, and Marc Swerts. 2014. Child--robot interaction across cultures: How does playing a game with a social robot compare to playing a game alone or with a friend? Computers in Human Behavior 40 (2014), 86--100.

Digital Library

[42]

David Harris Smith and Frauke Zeller. 2017. The Death and Lives of hitchBOT: The Design and Implementation of a Hitchhiking Robot. Leonardo 50, 1 (2017), 77--78.

[43]

Yutaka Suzuki, Lisa Galli, Ayaka Ikeda, Shoji Itakura, and Michiteru Kitazaki. 2015. Measuring empathy for human and robot hand pain using electroencephalography. Nature: Scientific Reports 5 (2015), 9.

[44]

Xiang Zhi Tan, Marynel Vázquez, Elizabeth J. Carter, Cecilia G. Morales, and Aaron Steinfeld. 2018. Inducing Bystander Interventions During Robot Abuse with Social Mechanisms. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI '18). ACM, New York, NY, USA, 169--177.

Digital Library

[45]

Andrea L. Thomaz and Cynthia Breazeal. 2008. Teachable robots: Understanding human teaching behavior to build more effective robot learners. Artificial Intelligence 172, 6--7 (2008), 716--737.

Digital Library

[46]

Andrea L. Thomaz, Guy Hoffman, and Cynthia Breazeal. 2005. Real-time interactive reinforcement learning for robots. AAAI 2005 workshop on human comprehensible machine learning. (2005).

[47]

Blay Whitby. 2008. Sometimes it's hard to be a robot: A call for action on the ethics of abusing artificial agents. Interacting with Computers 20, 3 (02 2008), 326--333.

Digital Library

[48]

David Whitney, Eric Rosen, James MacGlashan, Lawson L. S. Wong, and Stefanie Tellex. 2017. Reducing errors in object-fetching interactions through social feedback. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, New York, NY, USA, 1006--1013.

Digital Library

[49]

Sarah Woods, Kerstin Dautenhahn, and Joerg Schulz. 2004. The design space of robots: Investigating children's views. In RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No. 04TH8759). IEEE, New York, NY, USA, 47--52.

[50]

Ya-Huei Wu, Christine Fassert, and Anne-Sophie Rigaud. 2012. Designing robots for the elderly: appearance issue and beyond. Archives of Gerontology and Geriatrics 54, 1 (2012), 121--126.

[51]

Yuto Yamaji, Taisuke Miyake, Yuta Yoshiike, P. Ravindra De Silva, and Michio Okada. 2010. STB: Human-dependent Sociable Trash Box. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, New York, NY, USA, 197--198.

Cited By

Jarsve TJacobsen RSoma-Jestilä RWoźniak PNiess J(2024)AMoRE: Four Perspectives on Talking About and Experiencing RobotsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685342(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685342
Stemasov EWagner TAskari AJanek JRajabi OSchikorr AFrommel JGugenheimer JRukzio E(2024)DungeonMaker: Embedding Tangible Creation and Destruction in Hybrid Board Games through Personal Fabrication TechnologyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642243(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642243
Segate R(2024)Exposing, Reversing, and Inheriting Crimes as Traumas from the Neurosciences to Epigenetics: Why Criminal Law Cannot Yet Afford A(nother) Biology-induced OverhaulCriminal Justice Ethics10.1080/0731129X.2024.2376444(1-48)Online publication date: 24-Jul-2024
https://doi.org/10.1080/0731129X.2024.2376444
Show More Cited By

Index Terms

Punishable AI: Examining Users' Attitude Towards Robot Punishment
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Tactile and hand-based interfaces
      1. Haptic devices
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Social Robots in Learning Experiences of Adults with Intellectual Disability: An Exploratory Study
Human-Computer Interaction – INTERACT 2021
Abstract
The use of social robots has the potential to improve learning experiences in life skills for adults with intellectual disabilities (ID). Current research in the context of social robots in education has largely focused on how children with Autism ...
What Happens When Robots Punish? Evaluating Human Task Performance During Robot-Initiated Punishment

This article examines how people respond to robot-administered verbal and physical punishments. Human participants were tasked with sorting colored chips under time pressure and were punished by a robot when they made mistakes, such as inaccurate ...
Preventing Robot Abuse through Emotional Robot Responses
HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction

My research concerns group influence and prosocial behavior in a Human-Robot Interaction (HRI) context. My collaborators and I created and ran an experiment (N = 30) to measure if the emotional responses of a group of robots could induce participants to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems Conference

July 2020

2264 pages

ISBN:9781450369749

DOI:10.1145/3357236

General Chairs:
Ron Wakkary
Simon Fraser University, CA and Eindhoven University of Technology, NL
,
Kristina Andersen
Eindhoven University of Technology, NL
,
Program Chairs:
Will Odom
Simon Fraser University, CA
,
Audrey Desjardins
University of Washington, USA
,
Marianne Graves Petersen
Aarhus University, DK

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DIS '20

Sponsor:

SIGCHI

DIS '20: Designing Interactive Systems Conference 2020

July 6 - 10, 2020

Eindhoven, Netherlands

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Upcoming Conference

DIS '25

Sponsor:
sigchi

Designing Interactive Systems Conference

July 5 - 9, 2025

Funchal , Portugal

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
457
Total Downloads

Downloads (Last 12 months)79
Downloads (Last 6 weeks)12

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jarsve TJacobsen RSoma-Jestilä RWoźniak PNiess J(2024)AMoRE: Four Perspectives on Talking About and Experiencing RobotsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685342(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685342
Stemasov EWagner TAskari AJanek JRajabi OSchikorr AFrommel JGugenheimer JRukzio E(2024)DungeonMaker: Embedding Tangible Creation and Destruction in Hybrid Board Games through Personal Fabrication TechnologyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642243(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642243
Segate R(2024)Exposing, Reversing, and Inheriting Crimes as Traumas from the Neurosciences to Epigenetics: Why Criminal Law Cannot Yet Afford A(nother) Biology-induced OverhaulCriminal Justice Ethics10.1080/0731129X.2024.2376444(1-48)Online publication date: 24-Jul-2024
https://doi.org/10.1080/0731129X.2024.2376444
Zhao SZhao S(2024)Theoretical Preconditions of Criminal Imputation for Negligence Crime Involving AIPrinciple of Criminal Imputation for Negligence Crime Involving Artificial Intelligence10.1007/978-981-97-0722-5_2(25-57)Online publication date: 25-Feb-2024
https://doi.org/10.1007/978-981-97-0722-5_2
Rossmy BTerzimehić NDöring TBuschek DWiethoff A(2023)Point of no Undo: Irreversible Interactions as a Design StrategyProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581433(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581433
Scott ANeumann DNiess JWoźniak P(2023)Do You Mind? User Perceptions of Machine ConsciousnessProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581296(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581296
Oravec JOravec J(2022)Robo-Rage Against the Machine: Abuse, Sabotage, and Bullying of Robots and Autonomous VehiclesGood Robot, Bad Robot10.1007/978-3-031-14013-6_8(205-244)Online publication date: 1-Oct-2022
https://doi.org/10.1007/978-3-031-14013-6_8
LC RKaltenbrunner MMurer MWolf KOakley I(2021)NOW YOU SEE ME, NOW YOU DON'TProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3442448(1-7)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3442448
Lee MRuijten PFrank Lde Kort YIJsselsteijn WKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)People May Punish, But Not Blame RobotsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445284(1-11)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445284

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten