Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Open access

Learning and Personalizing Socially Assistive Robot Behaviors to Aid with Activities of Daily Living

Published: 24 October 2018 Publication History

Abstract

Socially assistive robots can autonomously provide activity assistance to vulnerable populations, including those living with cognitive impairments. To provide effective assistance, these robots should be capable of displaying appropriate behaviors and personalizing them to a user's cognitive abilities. Our research focuses on the development of a novel robot learning architecture that uniquely combines learning from demonstration (LfD) and reinforcement learning (RL) algorithms to effectively teach socially assistive robots personalized behaviors. Caregivers can demonstrate a series of assistive behaviors for an activity to the robot, which it uses to learn general behaviors via LfD. This information is used to obtain initial assistive state-behavior pairings using a decision tree. Then, the robot uses an RL algorithm to obtain a policy for selecting the appropriate behavior personalized to the user's cognition level. Experiments were conducted with the socially assistive robot Casper to investigate the effectiveness of our proposed learning architecture. Results showed that Casper was able to learn personalized behaviors for the new assistive activity of tea-making, and that combining LfD and RL algorithms significantly reduces the time required for a robot to learn a new activity.

References

[1]
D. Feil-Seifer and M. J. Matarić. 2005. Defining socially assistive robotics. In IEEE 9th International Conference on Rehabilitation Robotics. 465--468.
[2]
D. McColl and G. Nejat. 2013. Meal-time with a socially assistive robot and older adults at a long-term care facility. J. Human-Robot Interact. 2, 1 (2013), 152--171.
[3]
J. Li, W.-Y. G. Louie, S. Mohamed, F. Despond, and G. Nejat. 2016. A user-study with tangy the bingo facilitating robot and long-term care residents. In IEEE International Symposium on Robotics and Intelligent Sensors (IRIS’16). 109--115.
[4]
C. Thompson, S. Mohamed, G. Louie, J. C. He, J. Li, and G. Nejat. 2017. The robot tangy facilitating trivia games: A team- based user-study with long-term care residents. In IEEE International Symposium on Robotics and Intelligent Sensors (IRIS’17). 173--178.
[5]
A. Tapus, C. Tapus, and M. J. Mataric. 2008. User-robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intell. Serv. Robot 1, 2 (2008), 169--183.
[6]
K. Dautenhahn. 2003. Roles and functions of robots in human society: Implications from research in autism therapy. Robotica 21, 4 (2003), 443--452.
[7]
M. Heerink, B. Krose, V. Evers, and B. Wielinga. 2010. Assessing acceptance of assistive social agent technology by older adults: The Almere model. Int. J. Soc. Robot 2, 4 (2010), 361--375.
[8]
E. Torta, J. Oberzaucher, F. Werner, R. J. Cuijpers, and J. F. Juola. 2012. Attitudes towards socially assistive robots in intelligent homes: Results from laboratory studies and field trials. J. Human-Robot Interact. 1, 2 (2012), 76--99.
[9]
S. Andrist, X. Z. Tan, M. Gleicher, and B. Mutlu. 2014. Conversational gaze aversion for humanlike robots. In ACM/IEEE International Conference on Human-Robot Interaction. 25--32.
[10]
V. Ng-Thow-Hing, P. Luo, and S. Okita. 2010. Synchronized gesture and speech production for humanoid robots. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’10). 4617--4624.
[11]
C.-M. Huang and B. Mutlu. 2014. Learning-based modeling of multimodal behaviors for humanlike robots. In ACM/IEEE International Conference on Human-Robot Interaction. 57--64.
[12]
P. Liu, D. F. Glas, T. Kanda, H. Ishiguro, and N. Hagita. 2014. How to train your robot - teaching service robots to reproduce human social behavior. In IEEE International Symposium on Robot and Human Interactive Communication. 961--968.
[13]
A. H. Qureshi, Y. Nakamura, Y. Yoshikawa, and H. Ishiguro. 2016. Robot gains social intelligence through multimodal deep reinforcement learning. In IEEE-RAS 16th International Conference on Humanoid Robots. 745--751.
[14]
J. Hemminghaus and S. Kopp. 2017. Towards adaptive social behavior generation for assistive robots using reinforcement learning. In ACM/IEEE International Conference on Human-Robot Interaction. 332--340.
[15]
J. Chan and G. Nejat. 2012. Social intelligence for a robot engaging people in cognitive training activities. Int. J. Adv. Robot. Syst. 9, 4 (2012), 113.
[16]
W.-Y. G. Louie and G. Nejat. 2016. A learning from demonstration system architecture for robots learning social group recreational activities. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’16). 808--814.
[17]
Y. S. Chiang, T. S. Chu, C. D. Lim, T. Y. Wu, S. H. Tseng, and L. C. Fu. 2014. Personalizing robot behavior for interruption in social human-robot interaction. In IEEE Workshop on Advanced Robotics and Its Social Impacts (ARSO’14). 44--49.
[18]
D. Brooker and I. Latham. 2015. Person-Centred Dementia Care : Making Services Better with the VIPS Framework. Jessica Kingsley Publishers.
[19]
A. Tapus, C. Tapus, and M. J. Mataric. 2009. The use of socially assistive robots in the design of intelligent cognitive therapies for people with dementia. In IEEE International Conference on Rehabilitation Robotics. 924--929.
[20]
H. W. Park and A. M. Howard. 2015. Retrieving experience: Interactive instance-based learning methods for building robot companions. In IEEE International Conference on Robotics and Automation (ICRA’15). 6140--6145.
[21]
B. Mutlu, T. Kanda, J. Forlizzi, J. Hodgins, and H. Ishiguro. 2012. Conversational gaze mechanisms for humanlike robots. ACM Trans. Interact. Intell. Syst. 1, 2 (2012), 1--33.
[22]
A. D. Brenna, S. Chernova, M. Veloso, and B. Browning. 2009. A survey of robot learning from demonstration. Rob. Auton. Syst. 57, 5 (2009), 469--483.
[23]
S. Manschitz, J. Kober, M. Gienger, and J. Peters. 2015. Learning movement primitive attractor goals and sequential skills from kinesthetic demonstrations. Rob. Auton. Syst. 74, 5 (2015), 97--107.
[24]
J. J. Steil, F. Rothling, R. Haschke, and H. Ritter. 2004. Situated robot learning for multi-modal instruction and imitation of grasping. Rob. Auton. Syst. 47, 2 (2004), 129--141.
[25]
J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, and M. Kawato. 2004. Learning from demonstration and adaptation of biped locomotion. Rob. Auton. Syst. 47, 2 (2004), 79--91.
[26]
B. Kim, A. Massoud Farahmand, J. Pineau, and D. Precup. 2013. Learning from limited demonstrations. In Advances in Neural Information Processing Systems. 2859--2867.
[27]
C. Breazeal. 2003. Toward sociable robots. Rob. Auton. Syst. 42, 3 (2003), 167--175.
[28]
T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, G. Dulac-Arnold, I. Osband, J. Agapiou, J. Z. Leibo, and A. Gruslys. 2017. Learning from demonstrations for real world reinforcement learning. Arxiv Prepr. ArXiv1704.03732.
[29]
Alzheimer's Association. 2009. Memory Loss 8 10 Early Signs of Alzheimer's. Retrieved from https://www.alz.org/alzheimers_disease_10_signs_of_alzheimers.asp. Accessed December 22, 2017.
[30]
Alzheimer Society of Canada. 2017. 10 Warning Signs. Retrieved from http://www.alzheimer.ca/en/Home/About-dementia/Alzheimer-s-disease/10-warning-signs. Accessed December 22, 2017.
[31]
Alzheimer's Society. 2017. Symptoms. Retrieved from https://www.alzheimers.org.uk/info/20064/symptoms. Accessed December 22, 2017.
[32]
L. Breiman, J. Friedman, R. Olshen, and C. Stone. 1984. Classification and Regression Trees. Belmont, CA: Wadsworh 8 Brooks Cole.
[33]
B. Gupta. 2017. Analysis of various decision tree algorithms for classification in data mining. Int. J. Comput. Appl. 163, 8 (2017), 15--19.
[34]
S. Singh and M. Giri. 2014. Comparative study Id3, CART and C4. 5 decision tree algorithm: A survey. Int. J. Adv. Inf. Sci. Technol. 3, 7 (2014), 97--103.
[35]
H. J. Eysenck. 1991. Dimensions of personality: 16, 5 or 3?—Criteria for a taxonomic paradigm. Pers. Individ. Dif. 12, 8 (Jan. 1991), 773--790.
[36]
H. G. Wallbott. 1998. Bodily expression of emotion. Eur. J. Soc. Psychol. 28, 6 (1998), 879--896.
[37]
D. Morris. 1981. Gestures: Their Origins and Distribution. Stein 8 Day Pub.
[38]
R. S. Sutton and A. G. Barto. 1998. Introduction to Reinforcement Learning. MIT Press.
[39]
P. Auer, N. Cesa-Bianchi, and P. Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 2/3 (2002), 235--256.
[40]
R. Y. Chen, S. Sidor, P. Abbeel, and J. Schulman. 2017. UCB exploration via Q-ensembles. arXiv preprint arXiv:1706.01502.
[41]
P. Bovbel and G. Nejat. 2014. Casper: An assistive kitchen robot to promote aging in place. J. Med. Device. 8, 3, Article 30945 (2014).
[42]
Autonomous Systems and Biomechatronics Lab. 2013. Casper - Socially Assistive Humanoid Robot. Youtube. Retrieved from https://www.youtube.com/watch?v=noSJ9qWt_f0. Accessed May 16, 2017.
[43]
Amazon Web Services. 2017. Amazon Polly -- Lifelike Text-to-Speech. Retrieved from https://aws.amazon.com/polly/. Accessed November 9, 2017.
[44]
ROS. 2013. openni_tracker - ROS Wiki. Retrieved from http://wiki.ros.org/openni_tracker. Accessed November 24, 2017.
[45]
IBM Watson. 2017. Watson Speech to Text. Retrieved from https://www.ibm.com/watson/services/speech-to-text/. Accessed November 9, 2017.
[46]
S. Czarnuch and A. Mihailidis. 2011. The design of intelligent in-home assistive technologies: Assessing the needs of older adults with dementia and their caregivers. Gerontechnology 10, 3 (2011), 169--182.
[47]
M. C. Silveri, G. Reali, C. Jenner, and M. Puopolo. 2007. Attention and memory in the preclinical stage of dementia. J. Geriatr. Psychiatry Neurol. 20, 2 (2007), 67--75.
[48]
T. Fong, I. Nourbakhsh, and K. Dautenhahn. 2003. A survey of socially interactive robots. Rob. Auton. Syst. 42, 3--4 (2003), 143--166.
[49]
T. Tojo, Y. Matsusaka, T. Ishii, and T. Kobayashi. 2000. A conversational robot utilizing facial and body expressions. In IEEE International Conference on Systems, Man and Cybernetics. 858--863.
[50]
F. Ferland, D. Létourneau, A. Aumont, J. Frémy, M.-A. Legault, M. Lauria, and F. Michaud. 2012. Natural interaction design of a humanoid robot. J. Human-Robot Interact. 1, 2 (2012), 118--134.
[51]
E. T. Hall. 1966. The Hidden Dimension. New York: Doubleday 8 Co.
[52]
M. L. Walters, K. Dautenhahn, K. L. Koay, C. Kaouri, R. Boekhorst, C. Nehaniv, I. Werry, and D. Lee. 2005. Close encounters: Spatial distances between people and a robot of mechanistic appearance. In IEEE-RAS International Conference on Humanoid Robots. 450--455.
[53]
C. Rich, B. Ponsler, A. Holroyd, and C. L. Sidner. 2010. Recognizing engagement in human-robot interaction. In ACM/IEEE International Conference on Human-Robot Interaction. 375--382.

Cited By

View all
  • (2025)Enabling Embodied Human-Robot Co-Learning: Requirements, Method, and Test With Handover TaskIEEE Robotics and Automation Letters10.1109/LRA.2024.351987510:2(1425-1432)Online publication date: Feb-2025
  • (2024)Intelligent Robotics in Pediatric Cooperative Neurorehabilitation: A ReviewRobotics10.3390/robotics1303004913:3(49)Online publication date: 14-Mar-2024
  • (2024)Vision-Based Object Manipulation for Activities of Daily Living Assistance Using Assistive RobotAutomation10.3390/automation50200065:2(68-89)Online publication date: 15-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Human-Robot Interaction
ACM Transactions on Human-Robot Interaction  Volume 7, Issue 2
Special Issue on Artificial Intelligence and Human-Robot Interaction
July 2018
109 pages
EISSN:2573-9522
DOI:10.1145/3284682
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2018
Accepted: 01 August 2018
Received: 01 April 2018
Published in THRI Volume 7, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Human-robot interaction
  2. robot behavior learning
  3. socially assistive robots

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Canadian Consortium on Neurodegeneration in Aging
  • Ontario Graduate Scholarship (OGS) Program
  • AGE-WELL
  • Canada Research Chairs (CRC) Program

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)353
  • Downloads (Last 6 weeks)34
Reflects downloads up to 25 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Enabling Embodied Human-Robot Co-Learning: Requirements, Method, and Test With Handover TaskIEEE Robotics and Automation Letters10.1109/LRA.2024.351987510:2(1425-1432)Online publication date: Feb-2025
  • (2024)Intelligent Robotics in Pediatric Cooperative Neurorehabilitation: A ReviewRobotics10.3390/robotics1303004913:3(49)Online publication date: 14-Mar-2024
  • (2024)Vision-Based Object Manipulation for Activities of Daily Living Assistance Using Assistive RobotAutomation10.3390/automation50200065:2(68-89)Online publication date: 15-Apr-2024
  • (2024)Sensors, Techniques, and Future Trends of Human-Engagement-Enabled Applications: A ReviewAlgorithms10.3390/a1712056017:12(560)Online publication date: 6-Dec-2024
  • (2024)A Framework to Design Engaging Interactions in Socially Assistive Robots to Mitigate Dementia-Related SymptomsACM Transactions on Human-Robot Interaction10.1145/370088914:1(1-25)Online publication date: 18-Oct-2024
  • (2024)A Systematic Approach to Modeling Structured Behavior in Social RobotsProceedings of the 2024 International Symposium on Technological Advances in Human-Robot Interaction10.1145/3648536.3648540(29-37)Online publication date: 9-Mar-2024
  • (2024)Normative Requirements Operationalization with Large Language Models2024 IEEE 32nd International Requirements Engineering Conference (RE)10.1109/RE59067.2024.00022(129-141)Online publication date: 24-Jun-2024
  • (2024)Personalizing Activity Selection in Assistive Social Robots from Explicit and Implicit User FeedbackInternational Journal of Social Robotics10.1007/s12369-024-01124-2Online publication date: 9-Apr-2024
  • (2024)A Bayesian framework for learning proactive robot behaviour in assistive tasksUser Modeling and User-Adapted Interaction10.1007/s11257-024-09421-135:1Online publication date: 26-Dec-2024
  • (2024)Personalization of Child-Robot Interaction Through Reinforcement Learning and User ClassificationArtificial Intelligence for Neuroscience and Emotional Systems10.1007/978-3-031-61140-7_30(310-321)Online publication date: 31-May-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Full Access

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media