A Multi-Robot Architecture Framework for Effective Robot Teammates in Mixed-Initiative Teams
Authors: 
Matthias Scheutz, Bradley Oosterveld, John Peterson, Eric Wyss, and Evan KrauseAuthors Info & Claims
Matthias Scheutz, Bradley Oosterveld, John Peterson, Eric Wyss, and Evan KrauseAuthors Info & ClaimsMarch 2024
Pages 74 - 82
Abstract
Effective robotic teammates should be able to interact with humans in natural language about all task aspects, keep track of task and team states to coordinate their actions, and handle unexpected events autonomously. In this paper, we introduce a multi-robot architectural framework for effective robot teammates that allows robots to learn new tasks on the fly and monitor task execution to be able to detect unexpected faults and events. It enables robots to generate recovery plans, assess their effectiveness, and engage with human teammates in problem solving dialogues. We demonstrate the capabilities and operation of the framework in a complex mixed-initiative human-robot medical assembly and delivery task.
References
[1]
David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. 2014. Reducing the barrier to entry of complex robotic software: a moveit! case study. arXiv preprint arXiv:1404.3785 (2014).
[2]
Yan Ding, Xiaohan Zhang, Chris Paxton, and Shiqi Zhang. 2023. Task and Motion Planning with Large Language Models for Object Rearrangement. arXiv preprint arXiv:2303.06247v4 (2023).
[3]
Maria Fox and Derek Long. 2003. PDDL2.1: An extension to PDDL for expressing temporal planning domains. Journal of Artificial Intelligence Research 20 (2003), 61–124.
[4]
Tyler Frasca, Zhao Han, Jordan Allspaw, Holly Yanco, and Matthias Scheutz. 2020. Going cognitive: A demonstration of the utility of task-general cognitive architectures for adaptive robotic task performance. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 8110–8116.
[5]
Tyler Frasca, Bradley Oosterveld, Meia Chita-Tegmark, and Matthias Scheutz. 2021. Enabling fast instruction-based modification of learned robot skills. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 6075–6083.
[6]
Tyler Frasca, Bradley Oosterveld, Evan Krause, and Matthias Scheutz. 2018. One-Shot Interaction Learning from Natural Language Instruction and Demonstration. Advances in Cognitive Systems 6 (2018), 159–176.
[7]
Tyler Frasca and Matthias Scheutz. 2022. A Framework for Robot Self-Assessment of Expected Task Performance. IEEE Robotics and Automation Letters 7, 4 (2022), 12523–12530. https://doi.org/10.1109/LRA.2022.3219024
[8]
Zhao Han, Jordan Allspaw, Gregory LeMasurier, Jenna Parrillo, Daniel Giger, S Reza Ahmadzadeh, and Holly A Yanco. 2020. Towards mobile multi-task manipulation in a confined and integrated environment with irregular objects. In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 11025–11031.
[9]
Jamison Heard and Julian Fortune andJulie A. Adams. 2020. SAHRTA: A Supervisory-Based Adaptive Human-Robot Teaming Architecture. In IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA).
[10]
Jörg Hoffmann. 2003. The Metric-FF Planning System: Translating “Ignoring Delete Lists” to Numeric State Variables. Journal of Artificial Intelligence Research 20 (2003), 291–341.
[11]
James Kramer and Matthias Scheutz. 2007. Robotic Development Environments for Autonomous Mobile Robots: A Survey. Autonomous Robots 22, 2 (2007), 101–132.
[12]
Evan Krause, Michael Zillich, Thomas Williams, and Matthias Scheutz. 2014. Learning to recognize novel objects in one shot through human-robot interactions in natural language dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 28.
[13]
John E Laird. 2019. The Soar cognitive architecture. MIT press.
[14]
Microsoft Open SOurce. [n.d.]. Yet Another Revers Proxy. https://github.com/microsoft/reverse-proxy
[15]
Matthias Scheutz, Scott DeLoach, and Julie Adams. 2017. A Framework for developing and using shared mental models in human-agent teams. Journal of Cognitive Engineering and Decision Making 11, 3 (2017), 203–224. https://doi.org/10.1177/1555343416682891
[16]
Matthias Scheutz, Evan Krause, Brad Oosterveld, Tyler Frasca, and Robert Platt. 2017. Spoken Instruction-Based One-Shot Object and Action Learning in a Cognitive Robotic Architecture. In Proceedings of the 16th International Conference on Autoomous Agents and Multiagent Systems.
[17]
Matthias Scheutz, Thomas Williams, Evan Krause, Brley Oosterveld, Vasanth Sarathy, and Tyler Frasca. 2019. An Overview of the Distributed Integrated Affect and Reflection Cognitive DIARC Architecture. In Cognitive Architectures, Maria Isabel Aldinhas Ferreira, Joao Silva Sequeira, and Rodrigo Ventura (Eds.). Springer International Publishing.
[18]
Stanford Artificial Intelligence Laboratory et al.[n.d.]. Robotic Operating System. https://www.ros.org
[19]
Aaquib Tabrez, Matthew B. Luebbers, and Bradley Hayes. 2020. A Survey of Mental Modeling Techniques in Human–Robot Teaming. Current Robotics Reports 1 (2020), 259 – 267.
[20]
Robert Touchton, Daniel Kent, Tom Galluzzo, Carl D Crane III, David G Armstrong II, Nick Flann, Jeff Wit, and Phil Adsit. 2005. Planning and modeling extensions to the Joint Architecture for Unmanned Systems (JAUS) for application to unmanned ground vehicles. In Unmanned Ground Vehicle Technology VII, Vol. 5804. SPIE, 146–155.
[21]
J Gregory Trafton, Laura M Hiatt, Anthony M Harrison, Franklin P Tamborello, Sangeet S Khemlani, and Alan C Schultz. 2013. Act-r/e: An embodied cognitive architecture for human-robot interaction. Journal of Human-Robot Interaction 2, 1 (2013), 30–55.
[22]
Vaibhav V. Unhelkar, Shen Li, and Julie A. Shah. 2020. Decision-Making for Bidirectional Communication in Sequential Human-Robot Collaborative Tasks. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction. Association for Computing Machinery, New York, NY, USA, 329–341.
[23]
Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor. 2023. ChatGPT for Robotics: Design Principles and Model Abilities. Technical Report MSR-TR-2023-8. Microsoft. https://www.microsoft.com/en-us/research/publication/chatgpt-for-robotics-design-principles-and-model-abilities/
[24]
Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich. 2016. Fetch and freight: Standard platforms for service robot applications. In Workshop on autonomous mobile service robots. 1–6.
Index Terms
- A Multi-Robot Architecture Framework for Effective Robot Teammates in Mixed-Initiative Teams
Recommendations
Robot Task Planning for Mixed-Initiative Human Robot Interaction in Home Service Robot
CITWORKSHOPS '08: Proceedings of the 2008 IEEE 8th International Conference on Computer and Information Technology WorkshopsThe main reason of current robotics research is the difficulties of coping with dynamic environment, uncertainty, operational limitation, etc. Human robot interaction (HRI) becomes important role in robotics research and especially human’s roles in HRI ...
Lifelong Adaptation in Heterogeneous Multi-Robot Teams: Response to Continual Variation in Individual Robot Performance
Generating teams of robots that are able to perform their tasks over long periods of time requires the robots to be responsive to continual changes in robot team member capabilities and to changes in the state of the environment and mission. In this ...
Toward Initiative Decision-Making for Distributed Human-Robot Teams
HAI '18: Proceedings of the 6th International Conference on Human-Agent InteractionThe brief history of human-robot teams can be traced through the changing perspective of a robot's role within the team, which has evolved from being treated as a tool to a recent shift toward the desire to have the robot act as an equal partner. While ...
Comments
Information & Contributors
Information
Published In
March 2024
120 pages
ISBN:9798400716614
DOI:10.1145/3648536
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
TAHRI 2024
TAHRI 2024: 2024 International Symposium on Technological Advances in Human-Robot Interaction
March 9 - 10, 2024
CO, Boulder, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 66Total Downloads
- Downloads (Last 12 months)66
- Downloads (Last 6 weeks)37
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in