Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Bradykinesia Recognition in Parkinson’s Disease via Single RGB Video

Published: 09 February 2020 Publication History

Abstract

Parkinson’s disease is a progressive nervous system disorder afflicting millions of patients. Among its motor symptoms, bradykinesia is one of the cardinal manifestations. Experienced doctors are required for the clinical diagnosis of bradykinesia, but sometimes they also miss subtle changes, especially in early stages of such disease. Therefore, developing auxiliary diagnostic methods that can automatically detect bradykinesia has received more and more attention. In this article, we employ a two-stage framework for bradykinesia recognition based on the video of patient movement. First, convolution neural networks are trained to localize keypoints in each video frame. These time-varying coordinates form motion trajectories that represent the whole movement. From the trajectory, we then propose novel measurements, namely stability, completeness, and self-similarity, to quantify different motor behaviors. We also propose a periodic motion model called PMNet. An encoder--decoder structure is applied to learn a low dimensional representation of a motion process. The compressed motion process and quantified motor behaviors are combined as inputs to a fully-connected neural network. Different from the traditional means, our solution extends the application scenario outside the hospital and can be easily transplanted to conduct similar tasks. A commonly used clinical assessment is served as a case study. Experimental results based on real-world data validate the effectiveness of our approach for bradykinesia recognition.

References

[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system for large-scale machine learning. In Proceedings of USENIX Symposium on Operating Systems Design and Implementation, Vol. 16. 265--283.
[2]
Eduard Bakstein, Kevin Warwick, Jonathan Burgess, Øyvind Stavdahl, and Tipu Aziz. 2010. Features for detection of Parkinson’s disease tremor from local field potentials of the subthalamic nucleus. In International Conference on Cybernetic Intelligent Systems. 1--6.
[3]
Andrea Bandini, Silvia Orlandi, Hugo Jair Escalante, Fabio Giovannelli, Massimo Cincotta, Carlos A. Reyes-Garcia, Paola Vanni, Gaetano Zaccara, and Claudia Manfredi. 2017. Analysis of facial expressions in Parkinson’s disease through video-based automatic methods. Journal of Neuroscience Methods 281, 1 (2017), 7--20.
[4]
Andrea Bandini, Silvia Orlandi, Fabio Giovannelli, Andrea Felici, Massimo Cincotta, Daniela Clemente, Paola Vanni, Gaetano Zaccara, and Claudia Manfredi. 2016. Markerless analysis of articulatory movements in patients with Parkinson’s disease. Journal of Voice 30, 6 (2016), 766--e1.
[5]
J. Cancela, M. Pansera, M. T. Arredondo, J. J. Estrada, M. Pastorino, L. Pastor-Sanz, and J. L. Villalar. 2010. A comprehensive motor symptom monitoring and management system: The bradykinesia case. In International Conference of the IEEE Engineering in Medicine and Biology Society. 1008--1011.
[6]
François Chollet et al. 2015. Keras. Retrieved from https://github.com/fchollet/keras/.
[7]
Jonathan Chung, Sarah A. Chau, Nathan Herrmann, Krista L. Lanctôt, and Moshe Eizenman. 2018. Detection of apathy in alzheimer patients by analysing visual scanning behaviour with RNNs. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery 8 Data Mining. ACM, 149--157.
[8]
Bryan T. Cole, Serge H. Roy, and S. Hamid Nawab. 2011. Detecting freezing-of-gait during unscripted and unconstrained activity. In International Conference of the IEEE Engineering in Medicine and Biology Society. 5649--5652.
[9]
Esther Cubo, Pablo Martinez Martín, Jesus A. Martin-Gonzalez, Carmen Rodríguez-Blázquez, Jaime Kulisevsky, and ELEP Group Members. 2010. Motor laterality asymmetry and nonmotor symptoms in Parkinson’s disease. Movement Disorders 25, 1 (2010), 70--75.
[10]
Peng Dai, Femida Gwadry-Sridhar, Michael Bauer, and Michael Borrie. 2016. Bagging ensembles for the diagnosis and prognostication of alzheimer’s disease. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI’16). 3944--3950.
[11]
E. R. l. Dorsey, R. Constantinescu, J. P. Thompson, K. M. Biglan, R. G. Holloway, K. Kieburtz, F. J. Marshall, B. M. Ravina, G. Schifitto, A. Siderowf, and C. M. Tanner. 2007. Projected number of people with Parkinson disease in the most populous nations, 2005 through 2030. Neurology 68, 5 (2007), 384--386.
[12]
Saba Emrani, Anya McGuirk, and Wei Xiao. 2017. Prognosis and diagnosis of parkinson’s disease using multi-task learning. In Proceedings of International Conference on Knowledge Discovery and Data Mining. 1457--1466.
[13]
S. Fahn, R. L. Elton, and Members of the UPDRS Development Committee. 1987. Unified Parkinson’s disease rating scale. Recent Developments in Parkinson’s Disease 2 (1987), 153--164.
[14]
Valery L. Feigin, Amanuel Alemu Abajobir, Kalkidan Hassen Abate, Foad Abd-Allah, Abdishakur M. Abdulle, Semaw Ferede Abera, Gebre Yitayih Abyu, Muktar Beshir Ahmed, Amani Nidhal Aichour, Ibtihel Aichour, et al. 2017. Global, regional, and national burden of neurological disorders during 1990--2015: A systematic analysis for the Global Burden of Disease Study 2015. The Lancet Neurology 16, 11 (2017), 877--897.
[15]
W. R. Gibb and A. J. Lees. 1988. The relevance of the Lewy body to the pathogenesis of idiopathic Parkinson’s disease.Journal of Neurology, Neurosurgery 8 Psychiatry 51, 6 (1988), 745--752.
[16]
Christopher G. Goetz, Barbara C. Tilley, Stephanie R. Shaftman, Glenn T. Stebbins, Stanley Fahn, Pablo Martinez-Martin, Werner Poewe, Cristina Sampaio, Matthew B. Stern, Richard Dodel, Bruno Dubois, Robert Holloway, Joseph Jankovic, Jaime Kulisevsky, Anthony E. Lang, Andrew Lees, Sue Leurgans, Peter A. LeWitt, David Nyenhuis, C. Warren Olanow, Olivier Rascol, Anette Schrag, Jeanne A. Teresi, Jacobus J. van Hilten, and Nancy LaPelle. 2008. Movement disorder society-sponsored revision of the unified Parkinson’s disease rating scale (MDS-UPDRS): Scale presentation and clinimetric testing results. Movement Disorders 23, 15 (2008), 2129--2170.
[17]
Alex Graves. 2013. Generating sequences with recurrent neural networks. ArXiv e-prints (Aug. 2013). arxiv:cs/1308.0850
[18]
Margaret M. Hoehn and Melvin D. Yahr. 1998. Parkinsonism: Onset, progression, and mortality. Neurology 50, 2 (1998), 318--318.
[19]
Ji-Won Kim, Jae-Ho Lee, Yuri Kwon, Chul-Seung Kim, Gwang-Moon Eom, Seong-Beom Koh, Do-Young Kwon, and Kun-Woo Park. 2011. Quantification of bradykinesia during clinical finger taps using a gyrosensor in patients with Parkinson’s disease. Medical 8 Biological Engineering 8 Computing 49, 3 (2011), 365--371.
[20]
Xiaoli Liu, Peng Cao, André R. Gonçalves, Dazhe Zhao, and Arindam Banerjee. 2018. Modeling alzheimer’s disease progression with fused laplacian sparse group lasso. ACM Transactions on Knowledge Discovery from Data 12, 6 (2018), 65.
[21]
O. Martinez-Manzanera, E. Roosma, M. Beudel, R. W. K. Borgemeester, Teus van Laar, and Natasha M. Maurits. 2016. A method for automatic and objective scoring of bradykinesia using orientation sensors and classification algorithms. IEEE Transactions on Biomedical Engineering 63, 5 (2016), 1016--1024.
[22]
Ziad S. Nasreddine, Natalie A. Phillips, Valérie Bédirian, Simon Charbonneau, Victor Whitehead, Isabelle Collin, Jeffrey L. Cummings, and Howard Chertkow. 2005. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society 53, 4 (2005), 695--699.
[23]
Khalil Niazmand, Karin Tonn, Anastasios Kalaras, Urban M. Fietzek, Jan-Hinnerk Mehrkens, and Tim C. Lueth. 2011. Quantitative evaluation of Parkinson’s disease using sensor based smart glove. In International Symposium on Computer-Based Medical Systems. 1--8.
[24]
Khalil Niazmand, Karin Tonn, Anastasios Kalaras, S. Kammermeier, Kai Boetzel, Jan-Hinnerk Mehrkens, and Tim C. Lueth. 2011. A measurement device for motion analysis of patients with Parkinson’s disease using sensor based smart clothes. In International Conference on Pervasive Computing Technologies for Healthcare. 9--16.
[25]
K. Niazmand, K. Tonn, Y. Zhao, U. M. Fietzek, F. Schroeteler, K. Ziegler, A. O. Ceballos-Baumann, and T. C. Lueth. 2011. Freezing of Gait detection in Parkinson’s disease using accelerometer based smart clothes. In IEEE Biomedical Circuits and Systems Conference. 201--204.
[26]
M. Pastorino, J. Cancela, M. T. Arredondo, M. Pansera, L. Pastor-Sanz, F. Villagra, M. A. Pastor, and J. A. Martin. 2011. Assessment of bradykinesia in Parkinson’s disease patients through a multi-parametric system. In International Conference of the IEEE Engineering in Medicine and Biology Society. 1810--1813.
[27]
Werner Poewe, Klaus Seppi, Caroline M. Tanner, Glenda M. Halliday, Patrik Brundin, Jens Volkmann, Anette-Eleonore Schrag, and Anthony E. Lang. 2017. Parkinson disease. Nature Reviews Disease Primers 3, 1 (2017), 17013.
[28]
George Rigas, Alexandros T. Tzallas, Markos G. Tsipouras, Panagiota Bougia, Evanthia E. Tripoliti, Dina Baga, Dimitrios I. Fotiadis, Sofia G. Tsouli, and Spyridon Konitsiotis. 2012. Assessment of tremor activity in the Parkinson’s disease using a set of wearable sensors. IEEE Transactions on Information Technology in Biomedicine 16, 3 (2012), 478--487.
[29]
Arash Salarian, Heike Russmann, Christian Wider, Pierre R. Burkhard, Franios J. G. Vingerhoets, and Kamiar Aminian. 2007. Quantification of tremor and bradykinesia in Parkinson’s disease using a novel ambulatory monitoring system. IEEE Transactions on Biomedical Engineering 54, 2 (2007), 313--322.
[30]
Sofija Spasojević, Tihomir V. Ilić, Ivan Stojković, Veljko Potkonjak, Aleksandar Rodić, and José Santos-Victor. 2017. Quantitative assessment of the arm/hand Movements in Parkinson’s Disease Using a Wireless armband Device. Frontiers in Neurology 8 (2017), 388.
[31]
Nooritawati Md Tahir and Hany Hazfiza Manap. 2012. Parkinson disease gait classification based on machine learning approach. Journal of Applied Sciences 12, 2 (2012), 180--185.
[32]
Shaohua Teng, Naiqi Wu, Haibin Zhu, Luyao Teng, and Wei Zhang. 2018. SVM-DT-based adaptive and collaborative intrusion detection. IEEE/CAA Journal of Automatica Sinica 5, 1 (2018), 108--118.
[33]
PeiYun Zhang, Sheng Shu, and MengChu Zhou. 2018. An online fault detection model and strategies based on SVM-grid in clouds. IEEE/CAA Journal of Automatica Sinica 5, 2 (2018), 445--456.
[34]
Wenying Zhang, Huaguang Zhang, Jinhai Liu, Kai Li, Dongsheng Yang, and Hui Tian. 2017. Weather prediction with multiclass support vector machines in the fault detection of photovoltaic system. IEEE/CAA Journal of Automatica Sinica 4, 3 (2017), 520--525.
[35]
Christian Zimmermann and Thomas Brox. 2017. Learning to estimate 3D hand pose from single RGB images. In International Conference on Computer Vision.
[36]
Daphne G. M. Zwartjes, Tjitske Heida, Jeroen P. P. Van Vugt, Jan A. G. Geelen, and Peter H. Veltink. 2010. Ambulatory monitoring of activities and motor symptoms in Parkinson’s disease. IEEE Transactions on Biomedical Engineering 57, 11 (2010), 2778--2786.

Cited By

View all
  • (2024)When Is More Merrier? A Cloud-Based Architecture to Procure Impressions from Multiple Ad ExchangesInformation Systems Research10.1287/isre.2023.122135:1(294-317)Online publication date: 1-Mar-2024
  • (2023)A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitationJournal of NeuroEngineering and Rehabilitation10.1186/s12984-023-01186-920:1Online publication date: 2-May-2023
  • (2023)A Graded Assessment System for Parkinson’s Upper-Limb Bradykinesia Based on a Temporal Convolutional Network ModelIEEE Sensors Journal10.1109/JSEN.2023.332534423:23(29283-29292)Online publication date: 1-Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 14, Issue 2
April 2020
322 pages
ISSN:1556-4681
EISSN:1556-472X
DOI:10.1145/3382774
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 February 2020
Accepted: 01 October 2019
Revised: 01 October 2019
Received: 01 December 2018
Published in TKDD Volume 14, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Bradykinesia
  2. Parkinson’s disease
  3. RGB video
  4. computer vision
  5. time sequence analysis

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)66
  • Downloads (Last 6 weeks)19
Reflects downloads up to 27 Aug 2024

Other Metrics

Citations

Cited By

View all
  • (2024)When Is More Merrier? A Cloud-Based Architecture to Procure Impressions from Multiple Ad ExchangesInformation Systems Research10.1287/isre.2023.122135:1(294-317)Online publication date: 1-Mar-2024
  • (2023)A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitationJournal of NeuroEngineering and Rehabilitation10.1186/s12984-023-01186-920:1Online publication date: 2-May-2023
  • (2023)A Graded Assessment System for Parkinson’s Upper-Limb Bradykinesia Based on a Temporal Convolutional Network ModelIEEE Sensors Journal10.1109/JSEN.2023.332534423:23(29283-29292)Online publication date: 1-Dec-2023
  • (2023)Video and optoelectronics in movement disordersDigital Technologies in Movement Disorders10.1016/bs.irmvd.2023.05.003(227-244)Online publication date: 2023
  • (2021)A-WEAR Bracelet for Detection of Hand Tremor and Bradykinesia in Parkinson’s PatientsSensors10.3390/s2103098121:3(981)Online publication date: 2-Feb-2021
  • (2021)Remote Evaluation of Parkinson's Disease Using a Conventional Webcam and Artificial IntelligenceFrontiers in Neurology10.3389/fneur.2021.74265412Online publication date: 23-Dec-2021

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media