research-article

Open access

Learning to assess the quality of stroke rehabilitation exercises

Authors:

Daniel P. Siewiorek,

Asim Smailagic,

Alexandre Bernardino,

Sergi Bermúdez i BadiaAuthors Info & Claims

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Pages 218 - 228

https://doi.org/10.1145/3301275.3302273

Published: 17 March 2019 Publication History

Abstract

Due to the limited number of therapists, task-oriented exercises are often prescribed for post-stroke survivors as in-home rehabilitation. During in-home rehabilitation, a patient may become unmotivated or confused to comply prescriptions without the feedback of a therapist. To address this challenge, this paper proposes an automated method that can achieve not only qualitative, but also quantitative assessment of stroke rehabilitation exercises. Specifically, we explored a threshold model that utilizes the outputs of binary classifiers to quantify the correctness of a movements into a performance score. We collected movements of 11 healthy subjects and 15 post-stroke survivors using a Kinect sensor and ground truth scores from primary and secondary therapists. The proposed method achieves the following agreement with the primary therapist: 0.8436, 0.8264, and 0.7976 F1-scores on three task-oriented exercises. Experimental results show that our approach performs equally well or better than multi-class classification, regression, or the evaluation of the secondary therapist. Furthermore, we found a strong correlation (R² = 0.95) between the sum of computed exercise scores and the Fugl-Meyer Assessment scores, clinically validated motor impairment index of post-stroke survivors. Our results demonstrate a feasibility of automatically assessing stroke rehabilitation exercises with the decent agreement levels and clinical relevance.

Supplementary Material

MP4 File (p218-lee.mp4)

Download
337.67 MB

References

[1]

Richard A Berk. 2008. Classification and regression trees (CART). In Statistical learning from a regression perspective. Springer, 1--65.

[2]

Sandra A Billinger, Ross Arena, Julie Bernhardt, Janice J Eng, Barry A Franklin, Cheryl Mortag Johnson, Marilyn MacKay-Lyons, Richard F Macko, Gillian E Mead, Elliot J Roth, et al. 2014. Physical activity and exercise recommendations for stroke survivors: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 45, 8 (2014), 2532--2553.

[3]

Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg.

Digital Library

[4]

Antonio Bo, Mitsuhiro Hayashibe, and Philippe Poignet. 2011. Joint angle estimation in rehabilitation with inertial sensors and its integration with Kinect. In EMBC: Engineering in Medicine and Biology Conference. 3479--3483.

[5]

Helen Bourke-Taylor. 2003. Melbourne assessment of unilateral upper limb function: construct validity and correlation with the pediatric evaluation of disability inventory. Developmental medicine and child neurology 45, 2 (2003), 92--96.

[6]

Yao-Jen Chang, Wen-Ying Han, and Yu-Chi Tsai. 2013. A Kinect-based upper limb rehabilitation system to assist people with cerebral palsy. Research in developmental disabilities 34, 11 (2013), 3654--3659.

[7]

Roberto Colombo, Fabrizio Pisano, Alessandra Mazzone, Carmen Delconte, Silvestro Micera, M Chiara Carrozza, Paolo Dario, and Giuseppe Minuco. 2007. Design strategies to improve patient motivation during robot-aided rehabilitation. Journal of neuroengineering and rehabilitation 4, 1 (2007), 3.

[8]

Ricardo Cruz, Kelwin Fernandes, Joaquim F Pinto Costa, María Pérez Ortiz, and Jaime S Cardoso. 2018. Binary ranking for ordinal class imbalance. Pattern Analysis and Applications (2018), 1--9.

Digital Library

[9]

Teresa M Damush, Laurie Plue, Tamilyn Bakas, Arlene Schmid, and Linda S Williams. 2007. Barriers and facilitators to exercise among stroke survivors. Rehabilitation nursing 32, 6 (2007), 253--262.

[10]

Samarjit Das, Laura Trutoiu, Akihiko Murai, Dunbar Alcindor, Michael Oh, Fernando De la Torre, and Jessica Hodgins. 2011. Quantitative measurement of motor symptoms in Parkinson's disease: A study with full-body motion capture data. In Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE. IEEE, 6789--6792.

[11]

Pamela W Duncan, Richard Zorowitz, Barbara Bates, John Y Choi, Jonathan J Glasberg, Glenn D Graham, Richard C Katz, Kerri Lamberty, and Dean Reker. 2005. Management of adult stroke rehabilitation care: a clinical practice guideline. stroke 36, 9 (2005), e100--e143.

[12]

Eibe Frank and Mark Hall. 2001. A simple approach to ordinal classification. In European Conference on Machine Learning. Springer, 145--156.

Digital Library

[13]

Yoav Freund, Raj Iyer, Robert E Schapire, and Yoram Singer. 2003. An efficient boosting algorithm for combining preferences. Journal of machine learning research 4, Nov (2003), 933--969.

Digital Library

[14]

Pedro Antonio Gutierrez, Maria Perez-Ortiz, Javier Sanchez-Monedero, Francisco Fernandez-Navarro, and Cesar Hervas-Martinez. 2016. Ordinal regression methods: survey and experimental study. IEEE Transactions on Knowledge and Data Engineering 28, 1 (2016), 127--146.

Digital Library

[15]

Henk T Hendricks, Jacques van Limbeek, Alexander C Geurts, and Machiel J Zwarts. 2002. Motor recovery after stroke: a systematic review of the literature. Archives of physical medicine and rehabilitation 83, 11 (2002), 1629--1637.

[16]

Kevin Huang. 2015. Exploring In-Home Monitoring of Rehabilitation and Creating an Authoring Tool for Physical Therapists. Ph.D. Dissertation. Carnegie Mellon University.

[17]

Kevin Huang, Patrick J Sparto, Sara Kiesler, Asim Smailagic, Jennifer Mankoff, and Dan Siewiorek. 2014. A technology probe of wearable in-home computer-assisted physical therapy. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 2541--2550.

Digital Library

[18]

Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, et al. 2003. Technology probes: inspiring design for and with families. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 17--24.

Digital Library

[19]

Majid Janidarmian, Atena Roshan Fekr, Katarzyna Radecka, and Zeljko Zilic. 2014. Affordable erehabilitation monitoring platform. In Humanitarian Technology Conference-(IHTC), 2014 IEEE Canada International. IEEE, 1--6.

[20]

Michelle J Johnson, Xin Feng, Laura M Johnson, and Jack M Winters. 2007. Potential of a suite of robot/computer-assisted motivating systems for personalized, home-based, stroke rehabilitation. Journal of NeuroEngineering and Rehabilitation 4, 1 (2007), 6.

[21]

Peter Langhorne, Peter Sandercock, and Kameshwar Prasad. 2009. Evidence-based practice for stroke. The Lancet Neurology 8, 4 (2009), 308--309.

[22]

Ling Li and Hsuan-Tien Lin. 2007. Ordinal regression by extended binary classification. In Advances in neural information processing systems. 865--872.

Digital Library

[23]

Hsuan-Tien Lin and Ling Li. 2006. Large-margin thresholded ensembles for ordinal regression: Theory and practice. In International Conference on Algorithmic Learning Theory. Springer, 319--333

Digital Library

[24]

Roanna Lun and Wenbing Zhao. 2015. A survey of applications and human motion recognition with microsoft kinect. International Journal of Pattern Recognition and Artificial Intelligence 29, 05 (2015), 1555008.

[25]

Grace A MacDonald, Nicola M Kayes, and Felicity Bright. 2013. Barriers and facilitators to engagement in rehabilitation for people with stroke: a review of the literature. New Zealand Journal of Physiotherapy 41, 3 (2013), 112--121.

[26]

Niall Maclean, Pandora Pound, Charles Wolfe, and Anthony Rudd. 2000. Qualitative analysis of stroke patients' motivation for rehabilitation. Bmj 321, 7268 (2000), 1051--1054.

[27]

Margit Alt Murphy, Carin Willén, and Katharina S Sunnerhagen. 2011. Kinematic variables quantifying upper-extremity performance after stroke during reaching and drinking from a glass. Neurorehabilitation and neural repair 25, 1 (2011), 71--80.

[28]

Erienne V Olesh, Sergiy Yakovenko, and Valeriya Gritsenko. 2014. Automated assessment of upper extremity movement impairment due to stroke. PloS one 9, 8 (2014), e104487.

[29]

Ali Ozturk, Ahmet Tartar, Burcu Ersoz Huseyinsinoglu, and Ahmet H Ertas. 2016. A clinically feasible kinematic assessment method of upper extremity motor function impairment after stroke. Measurement 80 (2016), 207--216.

[30]

Jiann-I Pan, Hui-Wen Chung, and Jan-Jue Huang. 2013. Intelligent shoulder joint home-based self-rehabilitation monitoring system. Int. J. Smart Home 7, 5 (2013), 395--404.

[31]

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).

[32]

Tara S Patterson, MD Bishop, TE McGuirk, A Sethi, and LG Richards. 2011. Reliability of upper extremity kinematics while performing different tasks in individuals with stroke. Journal of motor behavior 43, 2 (2011), 121--130.

[33]

Bente Klarlund Pedersen and B Saltin. 2006. Evidence for prescribing exercise as therapy in chronic disease. Scandinavian journal of medicine & science in sports 16, S1 (2006), 3--63.

[34]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research 12, Oct (2011),2825--2830.

Digital Library

[35]

Bogdan Pogorelc, Zoran Bosnić, and Matjaž Gams. 2012. Automatic recognition of gait-related health problems in the elderly using machine learning. Multimedia Tools and Applications 58,2 (2012), 333--354.

Digital Library

[36]

Alexandra S Pollock, Lynn Legg, Peter Langhorne, and Cameron Sellars. 2000. Barriers to achieving evidence-based stroke rehabilitation. Clinical Rehabilitation 14,6 (2000),611--617.

[37]

Melinda Randall, John B Carlin, Patty Chondros, and Dinah Reddihough. 2001. Reliability of the Melbourne assessment of unilateral upper limb function. Developmental medicine and child neurology 43, 11 (2001), 761--767.

[38]

Denise Reid and Tasneem Hirji. 2004. The influence of a virtual reality leisure intervention program on the motivation of older adult stroke survivors: A pilot study. Physical & Occupational Therapy in Geriatrics 21,4 (2004),1--19.

[39]

Marijke Rensink, Marieke Schuurmans, Eline Lindeman, and Thora Hafsteinsdottir. 2009. Task-oriented training in rehabilitation after stroke: systematic review. Journal of advanced nursing 65, 4 (2009), 737--754.

[40]

Brandon Rohrer, Susan Fasoli, Hermano Igo Krebs, Richard Hughes, Bruce Volpe, Walter R Frontera, Joel Stein, and Neville Hogan. 2002. Movement smoothness changes during stroke recovery. Journal of Neuroscience 22,18 (2002), 8297--8304.

[41]

Julie Sanford, Julie Moreland, Laurie R Swanson, Paul W Stratford, and Carolyn Gowland. 1993. Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke. Physical therapy 73, 7 (1993), 447--454.

[42]

Daniel Siewiorek, Asim Smailagic, and Anind Dey. 2012. Architecture and applications of virtual coaches. Proc. IEEE 100, 8 (2012), 2472--2488.

[43]

César Souza, Andrew Kirillov, Diego Catalano, and Accord.NET contributors. 2014. The Accord.NET Framework.

[44]

Erik Stone and Marjorie Skubic. 2011. Evaluation of an inexpensive depth camera for in-home gait assessment. Journal of Ambient Intelligence and Smart Environments 3,4 (2011),349--361.

[45]

Chuan-Jun Su, Chang-Yu Chiang, and Jing-Yan Huang. 2014. Kinect-enabled home-based rehabilitation system using Dynamic Time Warping and fuzzy logic. Applied Soft Computing 22 (2014), 652--666.

Digital Library

[46]

Edward Taub, David M Morris, Jean Crago, Danna Kay King, Mary Bowman, Camille Bryson, Staci Bishop, Sonya Pearson, and Sharon E Shaw. 2011. Wolf motor function test (WMFT) manual. Birmingham: University of Alabama, CI Therapy Research Group (2011).

[47]

R PS Van Peppen, Gert Kwakkel, Sharon Wood-Dauphinee, H JM Hendriks, Ph J Van der Wees, and Joost Dekker. 2004. The impact of physical therapy on functional outcomes after stroke: what's the evidence? Clinical rehabilitation 18, 8 (2004), 833--862.

[48]

Bianca Zadrozny and Charles Elkan. 2002. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 694--699.

Digital Library

[49]

Liping Zhao, Xiong Lu, Xianglin Tao, and Xiaoli Chen. 2016. A Kinect-Based Virtual Rehabilitation System through Gesture Recognition. In Virtual Reality and Visualization (ICVRV), 2016 International Conference on. IEEE, 380--384.

[50]

Wenbing Zhao, Hai Feng, Roanna Lun, Deborah D Espy, and M Ann Reinthal. 2014. A Kinect-based rehabilitation exercise monitoring and guidance system. In Software Engineering and Service Science (ICSESS), 2014 5th IEEE International Conference on. IEEE, 762--765.

Cited By

Al-Ansari NAl-Thani DAl-Mansoori R(2024)User‐Centered Evaluation of Explainable Artificial Intelligence (XAI): A Systematic Literature ReviewHuman Behavior and Emerging Technologies10.1155/2024/46288552024:1Online publication date: 15-Jul-2024
https://doi.org/10.1155/2024/4628855
He TChen YWang LCheng H(2024)An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises AssessmentIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2024.340079032(1916-1925)Online publication date: 2024
https://doi.org/10.1109/TNSRE.2024.3400790
Murikipudi MAzmee AAl Hafiz Khan MPei Y(2024)CBSA: A Deep Transfer Learning Framework for Assessing Post-Stroke Exercises2024 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE)10.1109/CHASE60773.2024.00018(85-96)Online publication date: 19-Jun-2024
https://doi.org/10.1109/CHASE60773.2024.00018
Show More Cited By

Index Terms

Learning to assess the quality of stroke rehabilitation exercises
1. Applied computing
  1. Life and medical sciences
    1. Health care information systems
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Activity recognition and understanding
    2. Distributed artificial intelligence
      1. Intelligent agents

Recommendations

Gamification of stroke rehabilitation exercises using a smartphone
PervasiveHealth '14: Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare

Stroke is a disabling disease that requires extensive work of rehabilitation to improve the quality of life of patients. In order to increase the compliance and motivation of the patients, stroke rehabilitation exercises have been developed in a game-...
Towards customizable games for stroke rehabilitation
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Stroke is the leading cause of long term disability among adults in industrialized nations. The partial paralysis that stroke patients often experience can make independent living difficult or impossible. Research suggests that many of these patients ...
Muscle Extension Model for FES-Assisted Knee Swinging Ergometer for Stroke Patient
CICSYN '13: Proceedings of the 2013 Fifth International Conference on Computational Intelligence, Communication Systems and Networks

This work is focused on the development of physiological and mathematical model of muscle extension induced by FES-assisted knee swinging ergometer. A muscle model composed of muscle activation, muscle contraction and body segmental dynamics is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

March 2019

713 pages

ISBN:9781450362726

DOI:10.1145/3301275

General Chairs:
Wai-Tat Fu,
Shimei Pan,
Program Chairs:
Oliver Brdiczka,
Polo Chau,
Gaelle Calvary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI '19

Sponsor:

IUI '19: 24th International Conference on Intelligent User Interfaces

March 17 - 20, 2019

California, Marina del Ray

Acceptance Rates

IUI '19 Paper Acceptance Rate 71 of 282 submissions, 25%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
1,756
Total Downloads

Downloads (Last 12 months)358
Downloads (Last 6 weeks)46

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Al-Ansari NAl-Thani DAl-Mansoori R(2024)User‐Centered Evaluation of Explainable Artificial Intelligence (XAI): A Systematic Literature ReviewHuman Behavior and Emerging Technologies10.1155/2024/46288552024:1Online publication date: 15-Jul-2024
https://doi.org/10.1155/2024/4628855
He TChen YWang LCheng H(2024)An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises AssessmentIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2024.340079032(1916-1925)Online publication date: 2024
https://doi.org/10.1109/TNSRE.2024.3400790
Murikipudi MAzmee AAl Hafiz Khan MPei Y(2024)CBSA: A Deep Transfer Learning Framework for Assessing Post-Stroke Exercises2024 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE)10.1109/CHASE60773.2024.00018(85-96)Online publication date: 19-Jun-2024
https://doi.org/10.1109/CHASE60773.2024.00018
Sardari SSharifzadeh SDaneshkhah ALoke SPalade VDuncan MNakisa B(2024)LightPRA: A Lightweight Temporal Convolutional Network for Automatic Physical Rehabilitation Exercise AssessmentComputers in Biology and Medicine10.1016/j.compbiomed.2024.108382173(108382)Online publication date: May-2024
https://doi.org/10.1016/j.compbiomed.2024.108382
Chen TLee YChen TLee Y(2024)Smart Gait Healthcare Applications: Walking Status and Gait BiometricsSmart and Healthy Walking10.1007/978-3-031-59443-4_4(49-71)Online publication date: 1-May-2024
https://doi.org/10.1007/978-3-031-59443-4_4
Nakatake JArakawa HTajima TMiyazaki SChosa E(2023)Age- and sex-related differences in upper-body joint and endpoint kinematics during a drinking task in healthy adultsPeerJ10.7717/peerj.1657111(e16571)Online publication date: 19-Dec-2023
https://doi.org/10.7717/peerj.16571
Bajenaru LSorici AMocanu IFlorea AAntochi FRibigan A(2023)Shared Decision-Making to Improve Health-Related Outcomes for Adults with Stroke DiseaseHealthcare10.3390/healthcare1112180311:12(1803)Online publication date: 19-Jun-2023
https://doi.org/10.3390/healthcare11121803
Lee MChew C(2023)Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision MakingProceedings of the ACM on Human-Computer Interaction10.1145/36102187:CSCW2(1-22)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610218
Yao LLei QZhang HDu JGao S(2023)A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation ExercisesIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2023.331741131(3790-3802)Online publication date: 2023
https://doi.org/10.1109/TNSRE.2023.3317411
Rahman SSarker SHaque AUttsha MIslam MDeb S(2023)AI-Driven Stroke Rehabilitation Systems and Assessment: A Systematic ReviewIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2022.321908531(192-207)Online publication date: 2023
https://doi.org/10.1109/TNSRE.2022.3219085
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents