research-article

Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration

Authors:

Stefanos Nikolaidis,

Ariel D. Procaccia,

Siddhartha SrinivasaAuthors Info & Claims

HRI '17: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction

Pages 323 - 331

https://doi.org/10.1145/2909824.3020253

Published: 06 March 2017 Publication History

Abstract

In human-robot teams, humans often start with an inaccurate model of the robot capabilities. As they interact with the robot, they infer the robot's capabilities and partially adapt to the robot, i.e., they might change their actions based on the observed outcomes and the robot's actions, without replicating the robot's policy. We present a game-theoretic model of human partial adaptation to the robot, where the human responds to the robot's actions by maximizing a reward function that changes stochastically over time, capturing the evolution of their expectations of the robot's capabilities. The robot can then use this model to decide optimally between taking actions that reveal its capabilities to the human and taking the best action given the information that the human currently has. We prove that under certain observability assumptions, the optimal policy can be computed efficiently. We demonstrate through a human subject experiment that the proposed model significantly improves human-robot team performance, compared to policies that assume complete adaptation of the human to the robot.

References

[1]

J. Forlizzi, "How robotic products become social products: an ethnographic study of cleaning in the home," in Proceedings of the ACM/IEEE international conference on Human-robot interaction. ACM, 2007, pp. 129--136.

Digital Library

[2]

E. Cha, A. D. Dragan, and S. S. Srinivasa, "Perceived robot capability," in Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE International Symposium on. IEEE, 2015, pp. 541--548.

[3]

A. Powers and S. Kiesler, "The advisor robot: tracing people's mental model from a robot's physical attributes," in Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction. ACM, 2006, pp. 218--225.

Digital Library

[4]

V. Groom and C. Nass, "Can robots be teammates?: Benchmarks in human--robot teams," Interaction Studies, vol. 8, no. 3, pp. 483--500, 2007.

[5]

S. Nikolaidis, A. Kuznetsov, D. Hsu, and S. Srinivasa, "Formalizing human-robot mutual adaptation: A bounded memory model," in The Eleventh ACM/IEEE International Conference on Human Robot Interation. IEEE Press, 2016, pp. 75--82.

Digital Library

[6]

M. Tambe, Security and game theory: algorithms, deployed systems, lessons learned. Cambridge University Press, 2011.

Digital Library

[7]

B. D. Argall, S. Chernova, M. Veloso, and B. Browning, "A survey of robot learning from demonstration," Robot. Auton. Syst., 2009.

Digital Library

[8]

C. G. Atkeson and S. Schaal, "Robot learning from demonstration," in ICML, 1997.

Digital Library

[9]

P. Abbeel and A. Y. Ng, "Apprenticeship learning via inverse reinforcement learning," in ICML, 2004.

Digital Library

[10]

M. N. Nicolescu and M. J. Mataric, "Natural methods for robot task learning: Instructive demonstrations, generalization and practice," in AAMAS, 2003.

Digital Library

[11]

S. Chernova and M. Veloso, "Teaching multi-robot coordination using demonstration of communication and state sharing," in AAMAS, 2008.

Digital Library

[12]

B. Akgun, M. Cakmak, J. W. Yoo, and A. L. Thomaz, "Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective," in HRI, 2012.

Digital Library

[13]

S. Nikolaidis and J. Shah, "Human-robot cross-training: computational formulation, modeling and evaluation of a human team training strategy," in HRI, 2013.

Digital Library

[14]

O. Lemon and O. Pietquin, Data-Driven Methods for Adaptive Spoken Dialogue Systems: Computational Learning for Conversational Interfaces. Springer Publishing Company, Incorporated, 2012.

Digital Library

[15]

F. Broz, I. Nourbakhsh, and R. Simmons, "Designing pomdp models of socially situated tasks," in RO-MAN, 2011.

[16]

T. Bandyopadhyay, K. S. Won, E. Frazzoli, D. Hsu, W. S. Lee, and D. Rus, "Intention-aware motion planning," in WAFR. Springer, 2013.

[17]

O. Macindoe, L. P. Kaelbling, and T. Lozano-Pérez, "Pomcop: Belief space planning for sidekicks in cooperative games." in AIIDE, 2012.

Digital Library

[18]

S. Nikolaidis, R. Ramakrishnan, K. Gu, and J. Shah, "Efficient model learning from joint-action demonstrations for human-robot collaborative tasks," in HRI, 2015.

Digital Library

[19]

E. Karpas, S. J. Levine, P. Yu, and B. C. Williams, "Robust execution of plans for human-robot teams," in ICAPS, 2015.

Digital Library

[20]

T.-H. D. Nguyen, D. Hsu, W. S. Lee, T.-Y. Leong, L. P. Kaelbling, T. Lozano-Perez, and A. H. Grant, "Capir: Collaborative action planning with intention recognition." in AIIDE, 2011.

Digital Library

[21]

M. A. Goodrich and A. C. Schultz, "Human-robot interaction: a survey," Foundations and trends in human-computer interaction, 2007.

Digital Library

[22]

T. Kanda, T. Hirano, D. Eaton, and H. Ishiguro, "Interactive robots as social partners and peer tutors for children: A field trial," Human-computer interaction, 2004.

Digital Library

[23]

B. Robins, K. Dautenhahn, R. Te Boekhorst, and A. Billard, "Effects of repeated exposure to a humanoid robot on children with autism," in Designing a more inclusive world, 2004.

[24]

A. Green and H. Huttenrauch, "Making a case for spatial prompting in human-robot communication, in multimodal corpora: From multimodal behaviour theories to usable models," in workshop at LREC, 2006.

[25]

S. Ikemoto, H. B. Amor, T. Minato, B. Jung, and H. Ishiguro, "Physical human-robot interaction: Mutual learning and adaptation," IEEE Robot. Autom. Mag., 2012.

[26]

Y. Li, K. P. Tee, W. L. Chan, R. Yan, Y. Chua, and D. K. Limbu, "Role adaptation of human and robot in collaborative tasks," in 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2015, pp. 5602--5607.

[27]

D. Hadfield-Menell, A. Dragan, P. Abbeel, and S. Russell, "Cooperative inverse reinforcement learning," 2016.

[28]

P. Trautman and A. Krause, "Unfreezing the robot: Navigation in dense, interacting crowds," in Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on. IEEE, 2010, pp. 797--803.

[29]

D. Sadigh, S. Sastry, S. A. Seshia, and A. D. Dragan, "Planning for autonomous cars that leverages effects on human actions," in Proceedings of the Robotics: Science and Systems Conference (RSS), 2016.

[30]

V. Conitzer and T. Sandholm, "Computing the optimal strategy to commit to," in Proceedings of the 7th ACM conference on Electronic commerce. ACM, 2006, pp. 82--90.

Digital Library

[31]

M.-F. Balcan, A. Blum, N. Haghtalab, and A. D. Procaccia, "Commitment without regrets: Online learning in stackelberg security games," in Proceedings of the Sixteenth ACM Conference on Economics and Computation. ACM, 2015, pp. 61--78.

Digital Library

[32]

R. El Kaliouby and P. Robinson, "Real-time inference of complex mental states from facial expressions and head gestures," in Real-time vision for human-computer interaction. Springer, 2005, pp. 181--200.

[33]

S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. Pearson Education, 2003.

Digital Library

Cited By

Angelotti GChanel CMoreira Pinto ALounis CChauffaut CDrougard NDastani MSichman JAlechina NDignum V(2024)Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot TeamingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662852(58-67)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662852
Hou YCheon EJung MGrollman DBroadbent EJu WSoh HWilliams T(2024)Power in Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634949(269-282)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634949
van der Spaa LKober JGienger M(2024)Simultaneously learning intentions and preferences during physical human-robot cooperationAutonomous Robots10.1007/s10514-024-10167-348:4-5Online publication date: 4-Jun-2024
https://doi.org/10.1007/s10514-024-10167-3
Show More Cited By

Index Terms

Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration

Recommendations

Planning with Verbal Communication for Human-Robot Collaboration

Human collaborators coordinate effectively their actions through both verbal and non-verbal communication. We believe that the the same should hold for human-robot teams. We propose a formalism that enables a robot to decide optimally between taking a ...
Concurrent Probabilistic Motion Primitives for Obstacle Avoidance and Human-Robot Collaboration
Intelligent Robotics and Applications
Abstract
The paper proposed a new method to endow a robot with the ability of human-robot collaboration and online obstacle avoidance simultaneously. In other words, we construct a probabilistic model for human-robot collaboration primitives to learn the ...
Collision Driven Multi Scenario Approach for Human Collaboration with Industrial Robot
ICMRE 2018: Proceedings of the 2018 4th International Conference on Mechatronics and Robotics Engineering

This study focuses on the problem of robot interaction with a dynamically changing environment. Particular attention is paid to the problem of human collaboration with industrial robot in a shared common workspace. The paper identifies collisions and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '17: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction

March 2017

510 pages

ISBN:9781450343367

DOI:10.1145/2909824

General Chairs:
Bilge Mutlu
University of Wisconsin-Madison, USA
,
Manfred Tscheligi
University of Salzburg, Austria
,
Program Chairs:
Astrid Weiss
Vienna University of Technology, Austria
,
James E. Young
University of Manitoba, Canada

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HRI '17

Sponsor:

SIGAI
SIGCHI
IEEE-RAS

HRI '17: ACM/IEEE International Conference on Human-Robot Interaction

March 6 - 9, 2017

Vienna, Austria

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

60
Total Citations
View Citations
1,538
Total Downloads

Downloads (Last 12 months)240
Downloads (Last 6 weeks)11

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Angelotti GChanel CMoreira Pinto ALounis CChauffaut CDrougard NDastani MSichman JAlechina NDignum V(2024)Offline Risk-sensitive RL with Partial Observability to Enhance Performance in Human-Robot TeamingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662852(58-67)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662852
Hou YCheon EJung MGrollman DBroadbent EJu WSoh HWilliams T(2024)Power in Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634949(269-282)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634949
van der Spaa LKober JGienger M(2024)Simultaneously learning intentions and preferences during physical human-robot cooperationAutonomous Robots10.1007/s10514-024-10167-348:4-5Online publication date: 4-Jun-2024
https://doi.org/10.1007/s10514-024-10167-3
Choudhury ARay Akarmakar A(2024)Game Theoretic Model Based Human–Robot Collaboration in Waste MP (Multi-peripheral Imaging) Device Disassembly for RemanufacturingAdvances in Remanufacturing10.1007/978-3-031-52649-7_9(107-119)Online publication date: 30-Apr-2024
https://doi.org/10.1007/978-3-031-52649-7_9
Mandal DTriantafyllou SRadanovic GKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Performative reinforcement learningProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619396(23642-23680)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619396
Li YHan S(2023)Solving Strongly Convex and Smooth Stackelberg Games Without Modeling the Follower2023 American Control Conference (ACC)10.23919/ACC55779.2023.10156010(2332-2337)Online publication date: 31-May-2023
https://doi.org/10.23919/ACC55779.2023.10156010
Tian RTomizuka MDragan ABajcsy ACastellano GRiek LCakmak MLeite I(2023)Towards Modeling and Influencing the Dynamics of Human LearningProceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3568162.3578629(350-358)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3568162.3578629
Zhao YZhu Q(2023)Stackelberg Meta-Learning for Strategic Guidance in Multi-Robot Trajectory Planning2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS55552.2023.10342202(11342-11347)Online publication date: 1-Oct-2023
https://doi.org/10.1109/IROS55552.2023.10342202
Ge YSun HLv YWang JZhang SWang X(2023)Joint Planning of Intersection Trajectories and OD Routes under the Competition of CAV FirmsJournal of Transportation Engineering, Part A: Systems10.1061/JTEPBS.TEENG-7574149:7Online publication date: Jul-2023
https://doi.org/10.1061/JTEPBS.TEENG-7574
Parekh SLosey D(2023)Learning latent representations to co-adapt to humansAutonomous Robots10.1007/s10514-023-10109-547:6(771-796)Online publication date: 17-Jun-2023
https://doi.org/10.1007/s10514-023-10109-5
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents