research-article

Cognitive architecture for intuitive and interactive task learning in industrial collaborative robotics

Authors:

Francois Helenon,

Stephane Thiery,

Olivier GibaruAuthors Info & Claims

ICRCA 2021: 2021 the 5th International Conference on Robotics, Control and Automation

Pages 119 - 124

https://doi.org/10.1145/3471985.3472385

Published: 07 October 2021 Publication History

Abstract

This paper introduces a cognitive architecture, implemented in python3, designed with industrial collaborative robotics specifications in mind, to engage in a mixed-initiative teacher/learner setting called interactive task learning: a human can teach the robot, with natural and multimodal communication means, how to perform a task. The architecture has been built around explainable, modular representations (relational graphs and behavior trees) to ease the upgradability of the system and AI modules to adapt to realistic and complex settings. A first prototype based on speech and gesture communication means is proposed and has been validated on an industrial system to learn an unknown task. A link to a video of this validation is attached in the article.

References

[1]

Sandeep Ahluwalia. 2004. Hierarchical State Machines. In The Firmware Handbook.

[2]

Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, 2019. FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP. In Proceedings of the 2019 Conference of the North, Association for Computational Linguistics, Stroudsburg, PA, USA, 54–59.

[3]

John R. Anderson, Daniel Bothell, Michael D. Byrne, Scott Douglass, Christian Lebiere, 2004. An integrated theory of the mind. Psychological Review.

[4]

Michael Beetz, Daniel Bessler, Andrei Haidu, Mihai Pomarlan, Asil Kaan Bozcuoglu, 2018. Know Rob 2.0 - A 2nd Generation Knowledge Processing Framework for Cognition-Enabled Robotic Agents. In Proceedings - IEEE International Conference on Robotics and Automation.

[5]

Michael Beetz, Lorenz Mösenlechner, and Moritz Tenorth. 2010. CRAM - A Cognitive Robot Abstract Machine for everyday manipulation in human environments. In IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 - Conference Proceedings.

[6]

Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2018. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. (December 2018). Retrieved September 10, 2019 from http://arxiv.org/abs/1812.08008

[7]

Joyce Y. Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, 2018. Language to action: Towards interactive task learning with physical agents. In IJCAI International Joint Conference on Artificial Intelligence.

Digital Library

[8]

Michele Colledanchise. 2018. Behavior Trees in Robotics and Al.

Digital Library

[9]

Michael a Covington, Donald Nute, and André Vellino. 1995. Prolog programming in depth. Knowl. Creat. Diffus. Util. (1995).

Digital Library

[10]

Kutluhan Erol, James Hendler, and Dana S. Nau. 1994. HTN planning: Complexity and expressivity. In Proceedings of the National Conference on Artificial Intelligence.

Digital Library

[11]

Kevin French, Shiyu Wu, Tianyang Pan, Zheming Zhou, and Odest Chadwicke Jenkins. 2019. Learning behavior trees from demonstration. In Proceedings - IEEE International Conference on Robotics and Automation, Institute of Electrical and Electronics Engineers Inc., 7791–7797.

[12]

Kelleher R Guerin, Colin Lea, Chris Paxton, and Gregory D Hager. A Framework for End-User Instruction of a Robot Assistant for Manufacturing.

[13]

Francois Helenon, Laurent Bimont, Eric Nyiri, Stephane Thiery, and Olivier Gibaru. 2020. Learning prohibited and authorised grasping locations from a few demonstrations. In29th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2020.

[14]

Francois Felix Ingrand, Raja Chatila, Rachid Alami, and Frederic Robert. 1996. PRS: a high level supervision and control language for autonomous mobile robots. In Proceedings - IEEE International Conference on Robotics and Automation.

[15]

Iuliia Kotseruba and John K. Tsotsos. 2020. 40 years of cognitive architectures: core cognitive abilities and practical applications. Artif. Intell. Rev.53, 1 (July 2020), 17–94.

[16]

Unmesh Kurup and Christian Lebiere. 2012. What can cognitive architectures do for robotics? Biol. Inspired Cogn. Archit.2, (October 2012), 88–99.

[17]

John E. Laird, Kevin Gluck, John Anderson, Kenneth D. Forbus, Odest Chadwicke Jenkins, 2017. Interactive Task Learning. IEEE Intell. Syst.32, 4 (2017), 6–21.

Digital Library

[18]

Christian Lebiere. 2019. Act-R. In Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[19]

R.L. Lewis. 2001. Cognitive Theory: SOAR. In International Encyclopedia of the Social & Behavioral Sciences.

[20]

Natalia Lyubova. 2013. Developmental approach of perception for a humanoid robot. (2013), 170. Retrieved from https://tel.archives-ouvertes.fr/tel-00925067v1

[21]

Pooja Makula, Anurag Mishra, Akshay Kumar, Krit Karan, and V. K. Mittal. 2016. Multimodal smart robotic assistant. In Proceedings of 2015 International Conference on Signal Processing, Computing and Control, ISPCC 2015, Institute of Electrical and Electronics Engineers Inc., 18–23.

[22]

Gines Hidalgo Martinez, Yaadhav Raaj, Haroon Idrees, Donglai Xiang, Hanbyul Joo, 2019. Single-network whole-body pose estimation. In Proceedings of the IEEE International Conference on Computer Vision.

[23]

Shiwali Mohan. 2015. From Verbs to Tasks: An Integrated Account of Learning Tasks from Situated Interactive Instruction. ProQuest Diss. Theses (2015).

[24]

Anahita Mohseni-Kabir, Sonia Chernova, and Charles Rich. 2014. Collaborative learning of hierarchical task networks from demonstration and instruction. In AAAI Fall Symposium - Technical Report.

[25]

Anahita Mohseni-Kabir, Changshuo Li, Victoria Wu, Daniel Miller, Benjamin Hylak, 2019. Simultaneous learning of hierarchy and primitives for complex robot tasks. Auton. Robots (2019).

Digital Library

[26]

Clement Moulin-Frier, Tobias Fischer, Maxime Petit, Gregoire Pointeau, Jordi Ysard Puigbo, 2018. DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge about the World and the Self. IEEE Trans. Cogn. Dev. Syst. (2018).

[27]

Chris Paxton, Andrew Hundt, Felix Jonathan, Kelleher Guerin, and Gregory D. Hager. 2017. CoSTAR: Instructing collaborative robots with behavior trees and vision. In Proceedings - IEEE International Conference on Robotics and Automation.

[28]

Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, 2009. ROS: an open-source Robot Operating System. In ICRA workshop on open source software.

[29]

Ronald A. Rensink. 2000. Seeing, sensing, and scrutinizing. In Vision Research.

[30]

Charles Rich, Candace L. Sidner, and Neal Lesh. 2001. COLLAGEN: Applying collaborative discourse theory to human-computer interaction. AI Mag. (2001).

Digital Library

[31]

Francesco Rovida, Bjarne Grossmann, and Volker Kruger. 2017. Extended behavior trees for quick definition of flexible robotic tasks. In IEEE International Conference on Intelligent Robots and Systems.

[32]

Paul E. Rybski, Kevin Yoon, Jeremy Stolarz, and Manuela M. Veloso. 2007. Interactive robot task training through dialog and demonstration. In HRI 2007 - Proceedings of the 2007 ACM/IEEE Conference on Human-Robot Interaction - Robot as Team Member.

Digital Library

[33]

Moritz Tenorth and Michael Beetz. 2013. KnowRob: A knowledge processing infrastructure for cognition-enabled robots. Int. J. Rob. Res. (2013).

Digital Library

Cited By

Siwach GLi C(2024)Unveiling the Potential of Natural Language Processing in Collaborative Robots (Cobots): A Comprehensive Survey2024 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE59016.2024.10444393(1-6)Online publication date: 6-Jan-2024
https://doi.org/10.1109/ICCE59016.2024.10444393
Hélénon FThiery SNyiri EGibaru O(2024)A Cognitive Architecture for Human-Aware Interactive Robot Learning with Industrial Collaborative RobotsRobot 2023: Sixth Iberian Robotics Conference10.1007/978-3-031-58676-7_34(417-430)Online publication date: 27-Apr-2024
https://doi.org/10.1007/978-3-031-58676-7_34
Scherf LSchmidt APal SKoert D(2023)Interactively learning behavior trees from imperfect human demonstrationsFrontiers in Robotics and AI10.3389/frobt.2023.115259510Online publication date: 12-Jul-2023
https://doi.org/10.3389/frobt.2023.1152595
Show More Cited By

Index Terms

Cognitive architecture for intuitive and interactive task learning in industrial collaborative robotics
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
2. Computing methodologies
  1. Artificial intelligence
    1. Control methods
      1. Robotic planning
    2. Planning and scheduling
      1. Robotic planning
  2. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Cognitive Learning Methodologies for Brain-Inspired Cognitive Robotics

Cognitive robots are brain-inspired robots that are capable of inference, perception, and learning mimicking the cognitive mechanisms of the brain. Cognitive learning theories and methodologies for knowledge and behavior acquisition are centric in ...
Embodied Cognitive Robotics and the learning of sensorimotor schemes

Embodied Cognitive Robotics focuses its attention on the design of artificial agents capable of performing cognitive tasks autonomously. A central issue in this consists in studying process by which agents learn through interaction with their ...
Advances in Service and Industrial Robotics: Proceedings of the 27th International Conference on Robotics in Alpe-Adria Danube Region (RAAD 2018)

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICRCA 2021: 2021 the 5th International Conference on Robotics, Control and Automation

March 2021

129 pages

ISBN:9781450387484

DOI:10.1145/3471985

Copyright © 2021 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

AMVALOR
Région Hauts-de-France

Conference

ICRCA 2021

ICRCA 2021: 2021 the 5th International Conference on Robotics, Control and Automation

March 5 - 7, 2021

Seoul, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 15 of 29 submissions, 52%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
69
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Siwach GLi C(2024)Unveiling the Potential of Natural Language Processing in Collaborative Robots (Cobots): A Comprehensive Survey2024 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE59016.2024.10444393(1-6)Online publication date: 6-Jan-2024
https://doi.org/10.1109/ICCE59016.2024.10444393
Hélénon FThiery SNyiri EGibaru O(2024)A Cognitive Architecture for Human-Aware Interactive Robot Learning with Industrial Collaborative RobotsRobot 2023: Sixth Iberian Robotics Conference10.1007/978-3-031-58676-7_34(417-430)Online publication date: 27-Apr-2024
https://doi.org/10.1007/978-3-031-58676-7_34
Scherf LSchmidt APal SKoert D(2023)Interactively learning behavior trees from imperfect human demonstrationsFrontiers in Robotics and AI10.3389/frobt.2023.115259510Online publication date: 12-Jul-2023
https://doi.org/10.3389/frobt.2023.1152595
Wong PChui C(2022)Cognitive engine for augmented human decision-making in manufacturing process controlJournal of Manufacturing Systems10.1016/j.jmsy.2022.09.00765(115-129)Online publication date: Oct-2022
https://doi.org/10.1016/j.jmsy.2022.09.007

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents