short-paper

Public Access

An Interactive Learning and Adaptation Framework for Adaptive Robot Assisted Therapy

Authors:

Konstantinos Tsiakas,

Michalis Papakostas,

Benjamin Chebaa,

Vangelis Karkaletsis,

Fillia MakedonAuthors Info & Claims

PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

Article No.: 40, Pages 1 - 4

https://doi.org/10.1145/2910674.2935857

Published: 29 June 2016 Publication History

Abstract

In this paper, we present an interactive learning and adaptation framework. The framework combines Interactive Reinforcement Learning methods to effectively adapt and refine a learned policy to cope with new users. We argue that implicit feedback provided by the primary user and guidance from a secondary user can be integrated to the adaptation mechanism, resulting at a tailored and safe interaction. We illustrate this framework with a use case in Robot Assisted Therapy, presenting a Robot Yoga Trainer that monitors a yoga training session, adjusting the session parameters based on human motion activity recognition and evaluation through depth data, to assist the user complete the session, following a Reinforcement Learning approach.

References

[1]

J. Broekens. Emotion and reinforcement: affective facial expressions facilitate robot learning. In Artifical Intelligence for Human Computing. Springer, 2007.

Digital Library

[2]

F. Cruz et al. Interactive reinforcement learning through speech guidance in a domestic scenario. In Proc. IJCNN. IEEE, 2015.

[3]

T. Dobhal et al. Human activity recognition using binary motion image and deep learning. Procedia Computer Science, 2015.

[4]

J. Donahue et al. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.

[5]

N. Giullian, D. Ricks, A. Atherton, M. Colton, M. Goodrich, and B. Brinton. Detailed requirements for robots in autism therapy. In Proc SMC 2010. IEEE.

[6]

M. A. Goodrich et al. Incorporating a robot into an autism therapy team. IEEE Intelligent Systems, 2012.

Digital Library

[7]

J. Hernandez et al. Measuring the engagement level of tv viewers. In Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on. IEEE, 2013.

[8]

S. Janarthanam and O. Lemon. Adaptive generation in dialogue systems using dynamic user modeling. Computational Linguistics, 2014.

Digital Library

[9]

J. Karvonen and T. Vuorimaa. Heart rate and exercise intensity during sports activities. Sports Medicine, 1988.

[10]

W. B. Knox and P. Stone. Reinforcement learning from simultaneous human and mdp reward. In Proc. AAMAS, 2012.

Digital Library

[11]

A. Kolus, D. Imbeau, P.-A. Dubé, and D. Dubeau. Classifying work rate from heart rate measurements using an adaptive neuro-fuzzy inference system. Applied Ergonomics, 2016.

[12]

D. Lee, W.-h. Yun, C. Park, H. Yoon, and J. Kim. Analysis of children's posture for the bodily kinesthetic test. In Ubiquitous Robots and Ambient Intelligence (URAI), 2015 12th International Conference on. IEEE.

[13]

C. Li, Z. Rusák, I. Horváth, L. Ji, and Y. Hou. Current status of robotic stroke rehabilitation and opportunities for a cyber-physically assisted upper limb stroke rehabilitation. In Proceedings of the 10th international symposium on tools and methods of competitive engineering TMCE 2014. TMCE.

[14]

R. Loureiro, C. Collin, and W. Harwin. Robot aided therapy: challenges ahead for upper limb stroke rehabilitation. In Proceed of Intl Conf on Disability, Virtual Reality and Assoc Tech, 2004.

[15]

H. Modares, I. Ranatunga, F. L. Lewis, and D. O. Popa. Optimized assistive human--robot interaction using reinforcement learning. 2015.

[16]

R. Picard. Affective Computing. MIT Press, 1997.

Digital Library

[17]

A. Ramachandran and B. Scassellati. Developing adaptive social robot tutors for children. In 2015 AAAI Fall Symposium Series, 2015.

[18]

E. Senft, P. Baxter, and T. Belpaeme. Human-guided learning of social action selection for robot-assisted therapy. In 4th Workshop on Machine Learning for Interactive Systems, 2015.

[19]

H. Tanaka, K. D. Monahan, and D. R. Seals. Age-predicted maximal heart rate revisited. Journal of the American College of Cardiology, 2001.

[20]

A. Tapus, C. Tapus, and M. Matarić. Long term learning and online robot behavior adaptation for individuals with physical and cognitive impairments. In Field and Service Robotics. Springer, 2010.

[21]

M. E. Taylor and P. Stone. Transfer learning for reinforcement learning domains: A survey. The Journal of Machine Learning Research, 2009.

Digital Library

[22]

A. L. Thomaz and C. Breazeal. Reinforcement learning with human teachers: Evidence of feedback and guidance with implications for learning performance. In AAAI, 2006.

Digital Library

[23]

K. Tsiakas. Facilitating safe adaptation of interactive agents using interactive reinforcement learning. In Companion Publication of the 21st International Conference on Intelligent User Interfaces. ACM.

Digital Library

[24]

K. Tsiakas, M. Huber, and F. Makedon. A multimodal adaptive session manager for physical rehabilitation exercising. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 2015.

Digital Library

[25]

P. Wang et al. Deep convolutional neural networks for action recognition using depth map sequences. arXiv preprint arXiv:1501.04686, 2015.

[26]

C. Woodyard et al. Exploring the therapeutic effects of yoga and its ability to increase quality of life. International journal of yoga, 2011.

[27]

W. Zemzem and M. Tagina. A novel exploration/exploitation policy accelerating learning in both stationary and non-stationary environment navigation tasks. International Journal of Computer and Electrical Engineering, 2015.

Cited By

Ham J(2022)Influencing robot influenceInteraction Studies. Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies / Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies10.1075/is.00012.ham22:3(464-487)Online publication date: 28-Mar-2022
https://doi.org/10.1075/is.00012.ham
den Hengst FGrua Eel Hassouni AHoogendoorn M(2020)Reinforcement learning for personalization: A systematic literature reviewData Science10.3233/DS-2000283:2(107-147)Online publication date: 10-Apr-2020
https://doi.org/10.3233/DS-200028
Papakostas MKanal VAbujelala MTsiakas KMakedon FMakedon F(2019)Physical fatigue detection through EMG wearables and subjective user reportsProceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3316782.3322772(475-481)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3316782.3322772
Show More Cited By

An Interactive Learning and Adaptation Framework for Adaptive Robot Assisted Therapy
1. Computing methodologies

Recommendations

Interactive Learning and Adaptation for Robot Assisted Therapy for People with Dementia
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

In this paper, we present an adaptive cognitive music game designed to monitor and improve the attention levels of people with dementia. The goal of this game is to provide a customized protocol based on user needs and preferences, following the ...
Facilitating Safe Adaptation of Interactive Agents using Interactive Reinforcement Learning
IUI '16 Companion: Companion Publication of the 21st International Conference on Intelligent User Interfaces

In this paper, we propose a learning framework for the adaptation of an interactive agent to a new user. We focus on applications where safety and personalization are essential, as Rehabilitation Systems and Robot Assisted Therapy. We argue that ...
Environment Guided Interactive Reinforcement Learning: Learning from Binary Feedback in High-Dimensional Robot Task Environments
AAMAS '22: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems

Continuous Action-space Interactive Reinforcement learning (CAIR) is the first continuous action-space interactive reinforcement learning algorithm that can out-preform state-of-the-art reinforcement learning algorithms early on in training. We test ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

June 2016

455 pages

ISBN:9781450343374

DOI:10.1145/2910674

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

Conference

PETRA '16

PETRA '16: 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

June 29 - July 1, 2016

Island, Corfu, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
396
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)9

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ham J(2022)Influencing robot influenceInteraction Studies. Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies / Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies10.1075/is.00012.ham22:3(464-487)Online publication date: 28-Mar-2022
https://doi.org/10.1075/is.00012.ham
den Hengst FGrua Eel Hassouni AHoogendoorn M(2020)Reinforcement learning for personalization: A systematic literature reviewData Science10.3233/DS-2000283:2(107-147)Online publication date: 10-Apr-2020
https://doi.org/10.3233/DS-200028
Papakostas MKanal VAbujelala MTsiakas KMakedon FMakedon F(2019)Physical fatigue detection through EMG wearables and subjective user reportsProceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3316782.3322772(475-481)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3316782.3322772
Schneider SKummert F(2017)Exploring embodiment and dueling bandit learning for preference adaptation in human-robot interaction2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)10.1109/ROMAN.2017.8172476(1325-1331)Online publication date: Aug-2017
https://doi.org/10.1109/ROMAN.2017.8172476

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten