research-article

Crafting with a Robot Assistant: Use Social Cues to Inform Adaptive Handovers in Human-Robot Collaboration

Authors:

Akansel Cosgun,

Dana KulicAuthors Info & Claims

HRI '23: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

Pages 252 - 260

https://doi.org/10.1145/3568162.3576998

Published: 13 March 2023 Publication History

Abstract

We study human-robot handovers in a naturalistic collaboration scenario, where a mobile manipulator robot assists a person during a crafting session by providing and retrieving objects used for wooden piece assembly (functional activities) and painting (creative activities). We collect quantitative and qualitative data from 20 participants in a Wizard-of-Oz study, generating the Functional And Creative Tasks Human-Robot Collaboration dataset (the FACT HRC dataset), available to the research community. This work illustrates how social cues and task context inform the temporal-spatial coordination in human-robot handovers, and how human-robot collaboration is shaped by and in turn influences people's functional and creative activities.

Supplementary Material

ZIP File (hrifp1152aux.zip)

Including the operator's cheat-sheet, the participant's instruction sheet, the questionnaire, and a video summary. For more supporting materials please see the FACT HRC dataset.

Download
76.66 MB

MP4 File (hrifp1152.mp4)

Supplemental video

Download
75.89 MB

MP4 File (HRI23-fp1152.mp4)

Presentation video for paper ID 1152, title "Crafting with a Robot Assistant: Use Social Cues to Inform Adaptive Handovers in Human-Robot Collaboration". Presented by Leimin Tian.

Download
61.57 MB

References

[1]

Christoph Bartneck, Dana Kulić, Elizabeth Croft, and Susana Zoghbi. 2009. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International journal of social robotics, Vol. 1, 1 (2009), 71--81.

[2]

Valentin Bazarevsky, Ivan Grishchenko, Karthik Raveendran, Tyler Zhu, Fan Zhang, and Matthias Grundmann. 2020. Blazepose: On-device real-time body pose tracking. arXiv preprint arXiv:2006.10204 (2020).

[3]

Hildo Bijl, Thomas B Schön, Jan-Willem van Wingerden, and Michel Verhaegen. 2017. System identification through online sparse Gaussian process regression with input noise. IFAC Journal of Systems and Control, Vol. 2 (2017), 1--11.

[4]

Maya Cakmak, Siddhartha S Srinivasa, Min Kyung Lee, Sara Kiesler, and Jodi Forlizzi. 2011. Using spatial and temporal contrast for fluent robot-human hand-overs. In Proceedings of the 6th international conference on Human-robot interaction. 489--496.

Digital Library

[5]

Wesley P Chan, Matthew KXJ Pan, Elizabeth A Croft, and Masayuki Inaba. 2020. An affordance and distance minimization based method for computing object orientations for robot human handovers. International Journal of Social Robotics, Vol. 12, 1 (2020), 143--162.

[6]

Wesley P Chan, Tin Tran, Sara Sheikholeslami, and Elizabeth Croft. 2021. An Experimental Validation and Comparison of Reaching Motion Models for Unconstrained Handovers: Towards Generating Humanlike Motions for Human-Robot Handovers. In 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids). IEEE, 356--361.

[7]

Qing Feng, Ben Letham, Hongzi Mao, and Eytan Bakshy. 2020. High-dimensional contextual policy search with unknown context rewards using Bayesian optimization. Advances in Neural Information Processing Systems, Vol. 33 (2020), 22032--22044.

[8]

Elena Corina Grigore, Kerstin Eder, Anthony G Pipe, Chris Melhuish, and Ute Leonards. 2013. Joint action understanding improves robot-to-human object handover. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 4622--4629.

[9]

Kerry He, Pradeepsundar Simini, Wesley Chan, Dana Kulić, Elizabeth Croft, and Akansel Cosgun. 2022. On-The-Go Robot-to-Human Handovers with a Mobile Manipulator. In IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

Digital Library

[10]

Guy Hoffman. 2019. Evaluating fluency in human--robot collaboration. IEEE Transactions on Human-Machine Systems, Vol. 49, 3 (2019), 209--218.

[11]

Chien-ming Huang, Maya Cakmak, and Bilge Mutlu. 2015. Adaptive Coordination Strategies for Human-Robot Handovers. 2015 Robotics Science and Systems Conference (RSS 2015) (2015).

[12]

Nikos Karampatziakis, John Langford, and Paul Mineiro. 2020. Empirical likelihood for contextual bandits. Advances in Neural Information Processing Systems, Vol. 33 (2020), 9597--9607.

[13]

Andras Kupcsik, David Hsu, and Wee Sun Lee. 2018. Learning dynamic robot-to-human object handover from human feedback. In Robotics research. Springer, 161--176.

[14]

Guilherme J Maeda, Gerhard Neumann, Marco Ewerton, Rudolf Lioutikov, Oliver Kroemer, and Jan Peters. 2017. Probabilistic movement primitives for coordination of multiple human--robot collaborative tasks. Autonomous Robots, Vol. 41, 3 (2017), 593--612.

Digital Library

[15]

Eloise Matheson, Riccardo Minto, Emanuele GG Zampieri, Maurizio Faccio, and Giulio Rosati. 2019. Human--robot collaboration in manufacturing applications: A review. Robotics, Vol. 8, 4 (2019), 100.

[16]

Ajung Moon, Maneezhay Hashmi, HF Machiel Van Der Loos, Elizabeth A Croft, and Aude Billard. 2021. Design of Hesitation Gestures for Nonverbal Human-Robot Negotiation of Conflicts. ACM Transactions on Human-Robot Interaction (THRI), Vol. 10, 3 (2021), 1--25.

Digital Library

[17]

AJung Moon, Daniel M Troniak, Brian Gleeson, Matthew KXJ Pan, Minhua Zheng, Benjamin A Blumer, Karon MacLean, and Elizabeth A Croft. 2014. Meet me where i'm gazing: how shared attention gaze affects human-robot handover timing. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. 334--341.

Digital Library

[18]

Robin R Murphy and Debra Schreckenghost. 2013. Survey of metrics for human-robot interaction. In 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 197--198.

[19]

Heramb Nemlekar, Dharini Dutia, and Zhi Li. 2019. Object transfer point estimation for fluent human-robot handovers. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2627--2633.

Digital Library

[20]

Valerio Ortenzi, Akansel Cosgun, Tommaso Pardi, Wesley P Chan, Elizabeth Croft, and Dana Kulić. 2021. Object handovers: a review for robotics. IEEE Transactions on Robotics, Vol. 37, 6 (2021), 1855--1873.

[21]

Matthew KXJ Pan, Vidar Skjervøy, Wesley P Chan, Masayuki Inaba, and Elizabeth A Croft. 2017. Automated detection of handovers using kinematic features. The International Journal of Robotics Research, Vol. 36, 5--7 (2017), 721--738.

Digital Library

[22]

Sina Parastegari, Bahareh Abbasi, Ehsan Noohi, and Milovs Zefran. 2017. Modeling human reaching phase in human-human object handover with application in robot-human handover. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 3597--3602.

Digital Library

[23]

Joelle Pineau, Michael Montemerlo, Martha Pollack, Nicholas Roy, and Sebastian Thrun. 2003. Towards robotic assistants in nursing homes: Challenges and results. Robotics and autonomous systems, Vol. 42, 3--4 (2003), 271--281.

[24]

Nicole L Robinson, Teah Neal Hicks, Gavin Suddrey, and David J Kavanagh. 2020. The robot self-efficacy scale: robot self-efficacy, likability and willingness to interact increases after a robot-delivered tutorial. In IEEE/RSJ International Symposium on Robot and Human Interactive Communication 2020. IEEE, Institute of Electrical and Electronics Engineers, 272--277.

[25]

Rinat Rosenberg-Kima, Yaacov Koren, Maya Yachini, and Goren Gordon. 2019. Human-Robot-Collaboration (HRC): social robots as teaching assistants for training activities in small groups. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 522--523.

[26]

Leimin Tian and Sharon Oviatt. 2021. A taxonomy of social errors in human-robot interaction. ACM Transactions on Human-Robot Interaction (THRI), Vol. 10, 2 (2021), 1--32.

Digital Library

[27]

Antoine Toisoul, Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, and Maja Pantic. 2021. Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Machine Intelligence (2021). https://www.nature.com/articles/s42256-020-00280-0

[28]

Vaibhav V Unhelkar, Shen Li, and Julie A Shah. 2020. Decision-making for bidirectional communication in sequential human-robot collaborative tasks. In 2020 15th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 329--341.

Digital Library

[29]

Min Wu, Bertram Taetz, Ernesto Dickel Saraiva, Gabriele Bleser, and Steven Liu. 2019. On-line motion prediction and adaptive control in human-robot handover tasks. In 2019 IEEE International Conference on Advanced Robotics and its Social Impacts (ARSO). IEEE, 1--6.

Digital Library

[30]

Minhua Zheng, AJung Moon, Elizabeth A Croft, and Max Q-H Meng. 2015. Impacts of robot head gaze on robot-to-human handovers. International Journal of Social Robotics, Vol. 7, 5 (2015), 783--798. io

Cited By

Johansen SBrophy CRittenbruch MDonovan J(2024)Characterising CSCW Research on Human-Robot CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/36409998:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3640999
Yasar MIslam MIqbal TGrollman DBroadbent EJu WSoh HWilliams T(2024)PoseTron: Enabling Close-Proximity Human-Robot Collaboration Through Multi-human Motion PredictionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3635006(830-839)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3635006
Duan HYang YLi DWang P(2024)Human–robot object handover: Recent progress and future directionBiomimetic Intelligence and Robotics10.1016/j.birob.2024.1001454:1(100145)Online publication date: Mar-2024
https://doi.org/10.1016/j.birob.2024.100145

Index Terms

Crafting with a Robot Assistant: Use Social Cues to Inform Adaptive Handovers in Human-Robot Collaboration
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
      1. Robotic autonomy
2. Human-centered computing
  1. Collaborative and social computing
  2. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Collaborative interaction

Recommendations

Edutainment Software for the Pepper Robot
ISCSIC 2020: Proceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control

We present our software for the Pepper robot that is suitable for human-robot interaction in the area of edutainment. The combination of education and entertainment is achieved by modifying a state-of-the-art conversational artificial intelligence ...
Investigating the Impact of Human-Robot Collaboration on Creativity and Team Efficiency: A Case Study on Brainstorming in Presence of Robots
Social Robotics
Abstract
The main objective of this research is to explore the impact of deploying social robots in team-based intellectual cooperation, specifically during brainstorming sessions. A total of 72 participants (36 females and 36 males) were involved, with ...
Evaluating the augmented reality human-robot collaboration system

This article discusses an experimental comparison of three user interface techniques for interaction with a remotely located robot. A typical interface for such a situation is to teleoperate the robot using a camera that displays the robot's view of its ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '23: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

March 2023

631 pages

ISBN:9781450399647

DOI:10.1145/3568162

General Chairs:
Ginevra Castellano
Uppsala University, Sweden
,
Laurel Riek
University of California San Diego, USA
,
Program Chairs:
Maya Cakmak
University of Washington, USA
,
Iolanda Leite
KTH Royal Institute of Technology, Sweden

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council

Conference

HRI '23

Sponsor:

HRI '23: ACM/IEEE International Conference on Human-Robot Interaction

March 13 - 16, 2023

Stockholm, Sweden

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
395
Total Downloads

Downloads (Last 12 months)242
Downloads (Last 6 weeks)16

Reflects downloads up to 13 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Johansen SBrophy CRittenbruch MDonovan J(2024)Characterising CSCW Research on Human-Robot CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/36409998:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3640999
Yasar MIslam MIqbal TGrollman DBroadbent EJu WSoh HWilliams T(2024)PoseTron: Enabling Close-Proximity Human-Robot Collaboration Through Multi-human Motion PredictionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3635006(830-839)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3635006
Duan HYang YLi DWang P(2024)Human–robot object handover: Recent progress and future directionBiomimetic Intelligence and Robotics10.1016/j.birob.2024.1001454:1(100145)Online publication date: Mar-2024
https://doi.org/10.1016/j.birob.2024.100145
Pham TEspinosa-Leal L(2024)How Indirect and Direct Interaction Affect the Trustworthiness in Normal and Explainable Human-Robot InteractionSmart Technologies for a Sustainable Future10.1007/978-3-031-61905-2_40(411-422)Online publication date: 13-Jun-2024
https://doi.org/10.1007/978-3-031-61905-2_40

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents