Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Automatic Speech Classifier for Mild Cognitive Impairment and Early Dementia

Published: 15 October 2021 Publication History

Abstract

The World Health Organization estimates that 50 million people are currently living with dementia worldwide and this figure will almost triple by 2050. Current pharmacological treatments are only symptomatic, and drugs or other therapies are ineffective in slowing down or curing the neurodegenerative process at the basis of dementia. Therefore, early detection of cognitive decline is of the utmost importance to respond significantly and deliver preventive interventions. Recently, the researchers showed that speech alterations might be one of the earliest signs of cognitive defect, observable well in advance before other cognitive deficits become manifest. In this article, we propose a full automated method able to classify the audio file of the subjects according to the progress level of the pathology. In particular, we trained a specific type of artificial neural network, called autoencoder, using the visual representation of the audio signal of the subjects, that is, the spectrogram. Moreover, we used a data augmentation approach to overcome the problem of the large amount of annotated data usually required during the training phase, which represents one of the most major obstacles in deep learning. We evaluated the proposed method using a dataset of 288 audio files from 96 subjects: 48 healthy controls and 48 cognitively impaired participants. The proposed method obtained good classification results compared to the state-of-the-art neuropsychological screening tests and, with an accuracy of 90.57%, outperformed the methods based on manual transcription and annotation of speech.

References

[1]
Stefanie Abel, Walter Huber, and Gary S. Dell. 2009. Connectionist diagnosis of lexical disorders in aphasia. Aphasiology 23, 11 (2009), 1353–1378.
[2]
Emilia Ambrosini, Matteo Caielli, Marios Milis, Christos Loizou, Domenico Azzolino, Sarah Damanti, Laura Bertagnoli, Matteo Cesari, Sara Moccia, Manuel Cid, et al. 2019. Automatic speech analysis to early detect functional cognitive decline in elderly population. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 212–216.
[3]
Daniela Beltrami, Laura Calzà, Gloria Gagliardi, Enrico Ghidoni, Norina Marcello, Rema Rossini Favretti, and Fabio Tamburini. 2016. Automatic identification of mild cognitive impairment through the analysis of Italian spontaneous speech productions. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC’16). 2086–2093.
[4]
Daniela Beltrami, Gloria Gagliardi, Rema Rossini Favretti, Enrico Ghidoni, Fabio Tamburini, and Laura Calzà. 2018. Speech analysis by natural language processing techniques: A possible tool for very early detection of cognitive decline?Frontiers in Aging Neuroscience 10 (2018), 369.
[5]
Flavio Bertini, Giacomo Bergami, Danilo Montesi, Giacomo Veronese, Giulio Marchesini, and Paolo Pandolfi. 2018. Predicting frailty condition in elderly using multidimensional socioclinical databases. Proceedings of the IEEE 106, 4 (2018), 723–737.
[6]
Veronica Boschi, Eleonora Catricala, Monica Consonni, Cristiano Chesi, Andrea Moro, and Stefano F. Cappa. 2017. Connected speech in neurodegenerative language disorders: A review. Frontiers in Psychology 8 (2017), 269.
[7]
Andrew E. Budson and Paul R. Solomon. 2011. Memory Loss E-Book: A Practical Guide for Clinicians. Elsevier Health Sciences.
[8]
Laura Calzà, Daniela Beltrami, Gloria Gagliardi, Enrico Ghidoni, Norina Marcello, Rema Rossini-Favretti, and Fabio Tamburini. 2015. Should we screen for cognitive decline and dementia?Maturitas 82, 1 (2015), 28–35.
[9]
Laura Calzà, Gloria Gagliardi, Rema Rossini Favretti, and Fabio Tamburini. 2020. Linguistic features and automatic classifiers for identifying mild cognitive impairment and dementia. Computer Speech & Language 65 (2020), 101113.
[10]
Paola Ciurli, Paola Marangolo, and Anna Basso. 1996. Esame Del Linguaggio-II. OS. Retrieved on August 28, 2021 from https://www.giuntipsy.it/catalogo/test/esame-del-linguaggio-ii.
[11]
David Glenn Clark, Paula M. McLaughlin, Ellen Woo, Kristy Hwang, Sona Hurtz, Leslie Ramirez, Jennifer Eastman, Reshil-Marie Dukes, Puneet Kapur, Thomas P. DeRamus, et al. 2016. Novel verbal fluency scores and structural brain imaging for prediction of cognitive outcome in mild cognitive impairment. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring 2 (2016), 113–122.
[12]
Caroline Etienne, Guillaume Fidanza, Andrei Petrovskii, Laurence Devillers, and Benoit Schmauch. 2018. CNN+LSTM architecture for speech emotion recognition with data augmentation. arXiv:1802.05630. https://arxiv.org/abs/1802.05630
[13]
Sarah Tomaszewski Farias, Dan Mungas, Bruce R. Reed, Danielle Harvey, and Charles DeCarli. 2009. Progression of mild cognitive impairment to dementia in clinic- vs community-based cohorts. Archives of Neurology 66, 9 (2009), 1151–1157.
[14]
Marshal F. Folstein, Susan E. Folstein, and Paul R. McHugh. 1975. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 12, 3 (1975), 189–198.
[15]
Kristina Lundholm Fors, Kathleen C. Fraser, and Dimitrios Kokkinakis. 2018. Automated syntactic analysis of language abilities in persons with mild and subjective cognitive impairment. In MIE. 705–709.
[16]
K. Fraser, K. Lundholm Fors, Marie Eckerström, Charalambos Themistocleous, and Dimitrios Kokkinakis. 2018. Improving the sensitivity and specificity of MCI screening with linguistic information. In LREC Workshop: RaPID-2.
[17]
Kathleen C. Fraser, Kristina Lundholm Fors, and Dimitrios Kokkinakis. 2019. Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment. Computer Speech & Language 53 (2019), 121–139.
[18]
Kathleen C. Fraser, Kristina Lundholm Fors, Dimitrios Kokkinakis, and Arto Nordlund. 2017. An analysis of eye-movements during reading for the detection of mild cognitive impairment. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 1016–1026.
[19]
Kathleen C. Fraser, Kristina Lundholm Fors, Marie Eckerström, Fredrik Öhman, and Dimitrios Kokkinakis. 2019. Predicting MCI status from multimodal language data using cascaded classifiers. Frontiers in Aging Neuroscience 11 (2019), 205.
[20]
Michael Freitag, Shahin Amiriparian, Sergey Pugachevskiy, Nicholas Cummins, and Björn Schuller. 2017. auDeep: Unsupervised learning of representations from audio with deep recurrent neural networks. The Journal of Machine Learning Research 18, 1 (2017), 6340–6344.
[21]
Gábor Gosztolya, Veronika Vincze, László Tóth, Magdolna Pákáski, János Kálmán, and Ildikó Hoffmann. 2019. Identifying mild cognitive impairment and mild Alzheimer’s disease based on spontaneous speech using ASR and linguistic features. Computer Speech & Language 53 (2019), 181–197.
[22]
Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al. 2014. Deep speech: Scaling up end-to-end speech recognition. arXiv:1412.5567. https://arxiv.org/abs/1412.5567
[23]
Navdeep Jaitly and Geoffrey E. Hinton. 2013. Vocal tract length perturbation (VTLP) improves speech recognition. In Proceedings of the ICML Workshop on Deep Learning for Audio, Speech and Language, Vol. 117.
[24]
William Jarrold, Bart Peintner, David Wilkins, Dimitra Vergryi, Colleen Richey, Maria Luisa Gorno-Tempini, and Jennifer Ogar. 2014. Aided diagnosis of dementia type through computer-based analysis of spontaneous speech. In Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 27–37.
[25]
Naoyuki Kanda, Ryu Takeda, and Yasunari Obuchi. 2013. Elastic spectral distortion for low resource speech recognition with deep neural networks. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. IEEE, 309–314.
[26]
Chanwoo Kim, Ananya Misra, Kean Chin, Thad Hughes, Arun Narayanan, Tara Sainath, and Michiel Bacchiani. 2017. Generation of large-scale simulated utterances in virtual rooms to train deep-neural networks for far-field speech recognition in Google Home. Interspeech 2017 (2017), 379–383.
[27]
Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. Audio augmentation for speech recognition. In 16th Annual Conference of the International Speech Communication Association.
[28]
Alexandra Konig, Aharon Satt, Alex Sorin, Ran Hoory, Alexandre Derreumaux, Renaud David, and Phillippe H. Robert. 2018. Use of speech analyses within a mobile application for the assessment of cognitive impairment in elderly people. Current Alzheimer Research 15, 2 (2018), 120–129.
[29]
Alexandra König, Aharon Satt, Alexander Sorin, Ron Hoory, Orith Toledo-Ronen, Alexandre Derreumaux, Valeria Manera, Frans Verhey, Pauline Aalten, Phillipe H. Robert, et al. 2015. Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring 1, 1 (2015), 112–124.
[30]
Daniel M. Low, Kate H. Bentley, and Satrajit S. Ghosh. 2020. Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope Investigative Otolaryngology 5, 1 (2020), 96–116.
[31]
Xingchen Ma, Hongyu Yang, Qiang Chen, Di Huang, and Yunhong Wang. 2016. DepAudioNet: An efficient deep model for audio based depression classification. In Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. 35–42.
[32]
Juan J. G. Meilán, Francisco Martínez-Sánchez, Juan Carro, José A. Sánchez, and Enrique Pérez. 2012. Acoustic markers associated with impairment in language processing in Alzheimer’s disease. The Spanish Journal of Psychology 15, 2 (2012), 487–494.
[33]
Alex J. Mitchell. 2009. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. Journal of Psychiatric Research 43, 4 (2009), 411–431.
[34]
World Health Organization et al. 2017. Global action plan on the public health response to dementia 2017–2025. Retrieved on August 28, 2021 from https://www.who.int/publications/i/item/global-action-plan-on-the-public-health-response-to-dementia-2017---2025.
[35]
Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. 2019. SpecAugment: A simple data augmentation method for automatic speech recognition. arXiv:1904.08779. https://arxiv.org/abs/1904.08779
[36]
Ronald C. Petersen. 2011. Clinical practice. mild cognitive impairment.The New England Journal of Medicine 364, 23 (2011), 2227.
[37]
Anirudh Raju, Sankaran Panchapagesan, Xing Liu, Arindam Mandal, and Nikko Strom. 2018. Data augmentation for robust keyword spotting under playback interference. arXiv:1808.00563. https://arxiv.org/abs/1808.00563
[38]
Charalambos Themistocleous, Marie Eckerström, and Dimitrios Kokkinakis. 2018. Identification of mild cognitive impairment from speech in Swedish using deep sequential neural networks. Frontiers in Neurology 9 (2018), 975.
[39]
Charalambos Themistocleous, Dimitrios Kokkinakis, Marie Eckerström, Kathleen Fraser, and Kristina Lundholm Fors. [n.d.]. Effects of mild cognitive impairment on vowel duration. Retrieved on August 28, 2021 from https://gup.ub.gu.se/publication/270215?lang=en.
[40]
László Tóth, Ildikó Hoffmann, Gábor Gosztolya, Veronika Vincze, Gréta Szatlóczki, Zoltán Bánréti, Magdolna Pákáski, and János Kálmán. 2018. A speech recognition-based solution for the automatic detection of mild cognitive impairment from spontaneous speech. Current Alzheimer Research 15, 2 (2018), 130–138.
[41]
Veronika Vincze, Gábor Gosztolya, László Tóth, Ildikó Hoffmann, and Gréta Szatlóczki. 2016. Detecting mild cognitive impairment by exploiting linguistic information from transcripts. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Berlin, Germany, August 2016. Association for Computational Linguistics, 181–187.
[42]
Qiang Wei, Amy Franklin, Trevor Cohen, and Hua Xu. 2018. Clinical text annotation-What factors are associated with the cost of time?. In AMIA Annual Symposium Proceedings, Vol. 2018. American Medical Informatics Association, 1552.
[43]
Anders Wimo, Maëlenn Guerchet, Gemma-Claire Ali, Yu-Tzu Wu, A. Matthew Prina, Bengt Winblad, Linus Jönsson, Zhaorui Liu, and Martin Prince. 2017. The worldwide costs of dementia 2015 and comparisons with 2010. Alzheimer’s & Dementia 13, 1 (2017), 1–7.
[44]
Bea Yu, Thomas F. Quatieri, James R. Williamson, and James C. Mundt. 2015. Cognitive impairment prediction in the elderly based on vocal biomarkers. In 16th Annual Conference of the International Speech Communication Association.

Cited By

View all
  • (2024)Machine Learning Approaches for Dementia Detection Through Speech and Gait Analysis: A Systematic Literature ReviewJournal of Alzheimer's Disease10.3233/JAD-231459100:1(1-27)Online publication date: 25-Jun-2024
  • (2024)Smart Solutions for Detecting, Predicting, Monitoring, and Managing Dementia in the Elderly: A SurveyIEEE Access10.1109/ACCESS.2024.342196612(100026-100056)Online publication date: 2024
  • (2023)Applied Machine Learning Techniques to Diagnose Voice-Affecting Conditions and Disorders: Systematic Literature ReviewJournal of Medical Internet Research10.2196/4610525(e46105)Online publication date: 19-Jul-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing for Healthcare
ACM Transactions on Computing for Healthcare  Volume 3, Issue 1
January 2022
255 pages
EISSN:2637-8051
DOI:10.1145/3485154
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2021
Accepted: 01 May 2021
Revised: 01 April 2021
Received: 01 November 2020
Published in HEALTH Volume 3, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Dementia
  2. mild cognitive impairment
  3. classification
  4. speech data augmentation
  5. neural networks

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • OPLON
  • Ethical Committee of Azienda Ospedaliera Reggio Emilia

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)398
  • Downloads (Last 6 weeks)16
Reflects downloads up to 30 Aug 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Machine Learning Approaches for Dementia Detection Through Speech and Gait Analysis: A Systematic Literature ReviewJournal of Alzheimer's Disease10.3233/JAD-231459100:1(1-27)Online publication date: 25-Jun-2024
  • (2024)Smart Solutions for Detecting, Predicting, Monitoring, and Managing Dementia in the Elderly: A SurveyIEEE Access10.1109/ACCESS.2024.342196612(100026-100056)Online publication date: 2024
  • (2023)Applied Machine Learning Techniques to Diagnose Voice-Affecting Conditions and Disorders: Systematic Literature ReviewJournal of Medical Internet Research10.2196/4610525(e46105)Online publication date: 19-Jul-2023
  • (2023)Natural language processing techniques for studying language in pathological ageing: A scoping reviewInternational Journal of Language & Communication Disorders10.1111/1460-6984.1287059:1(110-122)Online publication date: 24-Mar-2023
  • (2023)A Robust Deep Neural Network Framework for the Detection of Dementia2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN)10.1109/ICPCSN58827.2023.00119(686-691)Online publication date: Jun-2023
  • (2023)Early Detection of Cognitive Decline Using Voice Assistant CommandsICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095825(1-5)Online publication date: 4-Jun-2023
  • (2023)Detecting Speech Abnormalities With a Perceiver-Based Sequence Classifier that Leverages a Universal Speech Model2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)10.1109/ASRU57964.2023.10389789(1-7)Online publication date: 16-Dec-2023
  • (2023)Dementia Speech Dataset Creation and Analysis in Indic Languages—A Pilot StudyIEEE Access10.1109/ACCESS.2023.333479011(130697-130718)Online publication date: 2023
  • (2023)Reading and lexical–semantic retrieval tasks outperforms single task speech analysis in the screening of mild cognitive impairment and Alzheimer's diseaseScientific Reports10.1038/s41598-023-36804-y13:1Online publication date: 15-Jun-2023
  • (2023)A hybrid mental health prediction model using Support Vector Machine, Multilayer Perceptron, and Random Forest algorithmsHealthcare Analytics10.1016/j.health.2023.1001853(100185)Online publication date: Nov-2023
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media