research-article

An automatic Alzheimer’s disease classifier based on spontaneous spoken English

Authors:

Flavio Bertini,

Gianluca Lutero,

Danilo MontesiAuthors Info & Claims

Volume 72, Issue C

https://doi.org/10.1016/j.csl.2021.101298

Published: 01 March 2022 Publication History

Abstract

According to the World Health Organization, the number of people suffering from dementia worldwide will grow to 150 million by mid-century, and Alzheimer’s disease is the most common form of dementia contributing to 60%–70% of cases. The problem is compounded by the fact that current pharmacologic treatments are only symptomatic, and therapies are ineffective in slow down or cure the degenerative process. An automatic and standardize classifier for Alzheimer’s disease is thereby extremely important to rapidly respond and deliver as preventive as possible interventions. Speech alterations might be one of the earliest signs of cognitive defect and, recently, the researchers showed that they can be observable well in advance other cognitive deficits become manifest. In this paper, we propose a full automated method able to classify the spontaneous spoken production of the subjects. In particular, we trained an artificial neural network using the spectrogram of the audio signal, which is the visual representation of the speech of the subject. Moreover, to overcome the problem of the large amount of annotated data usually required for training deep learning models, we used a specific data augmentation approach that avoids distorting the original samples. We evaluated the proposed method using the English Pitt Corpus from DementiaBank. The used dataset consists of 180 subjects: 43 healthy controls and 137 Alzheimer’s disease patients. The proposed method outperformed the other approaches in the literature based on manual and semi-automatic transcription and annotation of speech, improving the classification capability by 5.93%, and obtained good classification results compared to the state-of-the-art neuropsychological screening tests (i.e., the Mini-Mental State Examination and the Activities of Daily Living portion of the Blessed Dementia Rating Scale) exhibiting an accuracy of 93.30% and an F1 score of 88.50%.

References

[1]

Abbott A., Dementia: a problem for our age, Nature 475 (7355) (2011) S2–S4.

[2]

Abel S., Huber W., Dell G.S., Connectionist diagnosis of lexical disorders in aphasia, Aphasiology 23 (11) (2009) 1353–1378.

[3]

Ambrosini E., Caielli M., Milis M., Loizou C., Azzolino D., Damanti S., Bertagnoli L., Cesari M., Moccia S., Cid M., et al., Automatic speech analysis to early detect functional cognitive decline in elderly population, in: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, IEEE, 2019, pp. 212–216.

[4]

Balagopalan A., Eyre B., Rudzicz F., Novikova J., To BERT or not to BERT: Comparing speech and language-based approaches for Alzheimer’s disease detection, 2020, arXiv preprint arXiv:2008.01551.

[5]

Becker J.T., Boiler F., Lopez O.L., Saxton J., McGonigle K.L., The natural history of Alzheimer’s disease: description of study cohort and accuracy of diagnosis, Arch. Neurol. 51 (6) (1994) 585–594.

[6]

Becker J.T., Boller F., Saxton J., McGonigle-Gibson K.L., Normal rates of forgetting of verbal and non-verbal material in Alzheimer’s disease, Cortex: A J. Devoted Stud. Nervous Syst. Behav. (1987).

[7]

Beltrami, D., Calzà, L., Gagliardi, G., Ghidoni, E., Marcello, N., Favretti, R.R., Tamburini, F., 2016. Automatic identification of mild cognitive impairment through the analysis of Italian spontaneous speech productions. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation. LREC’16. pp. 2086–2093.

[8]

Beltrami D., Gagliardi G., Rossini Favretti R., Ghidoni E., Tamburini F., Calzà L., Speech analysis by natural language processing techniques: a possible tool for very early detection of cognitive decline?, Front. Aging Neurosci. 10 (2018) 369.

[9]

Bertini F., Bergami G., Montesi D., Veronese G., Marchesini G., Pandolfi P., Predicting frailty condition in elderly using multidimensional socioclinical databases, Proc. IEEE 106 (4) (2018) 723–737.

[10]

Boschi V., Catricala E., Consonni M., Chesi C., Moro A., Cappa S.F., Connected speech in neurodegenerative language disorders: a review, Front. Psychol. 8 (2017) 269.

[11]

Budson A.E., Solomon P.R., Memory Loss E-Book: A Practical Guide for Clinicians, Elsevier Health Sciences, 2011.

[12]

Calzà L., Beltrami D., Gagliardi G., Ghidoni E., Marcello N., Rossini-Favretti R., Tamburini F., Should we screen for cognitive decline and dementia?, Maturitas 82 (1) (2015) 28–35.

[13]

Calzà L., Gagliardi G., Favretti R.R., Tamburini F., Linguistic features and automatic classifiers for identifying mild cognitive impairment and dementia, Comput. Speech Lang. 65 (2020).

[14]

Clark D.G., McLaughlin P.M., Woo E., Hwang K., Hurtz S., Ramirez L., Eastman J., Dukes R.-M., Kapur P., DeRamus T.P., et al., Novel verbal fluency scores and structural brain imaging for prediction of cognitive outcome in mild cognitive impairment, Alzheimer’s Dement. 2 (2016) 113–122.

[15]

Etienne C., Fidanza G., Petrovskii A., Devillers L., Schmauch B., Cnn+ lstm architecture for speech emotion recognition with data augmentation, 2018, arXiv preprint arXiv:1802.05630.

[16]

Farias S.T., Mungas D., Reed B.R., Harvey D., DeCarli C., Progression of mild cognitive impairment to dementia in clinic-vs community-based cohorts, Arch. Neurol. 66 (9) (2009) 1151–1157.

[17]

Fors K.L., Fraser K.C., Kokkinakis D., Automated syntactic analysis of language abilities in persons with mild and subjective cognitive impairment, in: MIE, 2018, pp. 705–709.

[18]

Fraser K.C., Fors K.L., Kokkinakis D., Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment, Comput. Speech Lang. 53 (2019) 121–139.

[19]

Fraser, K.C., Fors, K.L., Kokkinakis, D., Nordlund, A., 2017. An analysis of eye-movements during reading for the detection of mild cognitive impairment. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pp. 1016–1026.

[20]

Fraser K.C., Lundholm Fors K., Eckerström M., Öhman F., Kokkinakis D., Predicting MCI status from multimodal language data using cascaded classifiers, Front. Aging Neurosci. 11 (2019) 205.

[21]

Fraser, K., Lundholm Fors, K., Eckerström, M., Themistocleous, C., Kokkinakis, D., 2018. Improving the sensitivity and specificity of MCI screening with linguistic information. In: LREC Workshop: RaPID-2. Miyazaki, Japan.

[22]

Freitag M., Amiriparian S., Pugachevskiy S., Cummins N., Schuller B., audeep: Unsupervised learning of representations from audio with deep recurrent neural networks, J. Mach. Learn. Res. 18 (1) (2017) 6340–6344.

[23]

Goodglass H., Kaplan E., Weintraub S., BDAE: The Boston Diagnostic Aphasia Examination, Lippincott Williams & Wilkins Philadelphia, PA, 2001.

[24]

Gosztolya G., Vincze V., Tóth L., Pákáski M., Kálmán J., Hoffmann I., Identifying mild cognitive impairment and mild alzheimer’s disease based on spontaneous speech using ASR and linguistic features, Comput. Speech Lang. 53 (2019) 181–197.

[25]

Haider F., De La Fuente S., Luz S., An assessment of paralinguistic acoustic features for detection of Alzheimer’s dementia in spontaneous speech, IEEE J. Sel. Top. Sign. Proces. 14 (2) (2019) 272–281.

[26]

Handels R.L., Wolfs C.A., Aalten P., Joore M.A., Verhey F.R., Severens J.L., Diagnosing Alzheimer’s disease: a systematic review of economic evaluations, Alzheimer’s Dement. 10 (2) (2014) 225–237.

[27]

Hannun A., Case C., Casper J., Catanzaro B., Diamos G., Elsen E., Prenger R., Satheesh S., Sengupta S., Coates A., et al., Deep speech: Scaling up end-to-end speech recognition, 2014, arXiv preprint arXiv:1412.5567.

[28]

Jaitly, N., Hinton, G.E., 2013. Vocal tract length perturbation (VTLP) improves speech recognition. In: Proc. ICML Workshop on Deep Learning for Audio, Speech and Language, Vol. 117.

[29]

Jarrold, W., Peintner, B., Wilkins, D., Vergryi, D., Richey, C., Gorno-Tempini, M.L., Ogar, J., 2014. Aided diagnosis of dementia type through computer-based analysis of spontaneous speech. In: Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal To Clinical Reality. pp. 27–37.

[30]

Kalaria R.N., Maestre G.E., Arizaga R., Friedland R.P., Galasko D., Hall K., Luchsinger J.A., Ogunniyi A., Perry E.K., Potocnik F., et al., Alzheimer’s disease and vascular dementia in developing countries: prevalence, management, and risk factors, Lancet Neurol. 7 (9) (2008) 812–826.

[31]

Kanda N., Takeda R., Obuchi Y., Elastic spectral distortion for low resource speech recognition with deep neural networks, in: 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, IEEE, 2013, pp. 309–314.

[32]

Kim C., Misra A., Chin K., Hughes T., Narayanan A., Sainath T., Bacchiani M., Generation of large-scale simulated utterances in virtual rooms to train deep-neural networks for far-field speech recognition in Google Home, 2017.

[33]

Ko, T., Peddinti, V., Povey, D., Khudanpur, S., 2015. Audio augmentation for speech recognition. In: Sixteenth Annual Conference of the International Speech Communication Association.

[34]

Kong W., Jang H., Carenini G., Field T.S., Exploring neural models for predicting dementia from language, Comput. Speech Lang. 68 (2021).

[35]

Konig A., Satt A., Sorin A., Hoory R., Derreumaux A., David R., Robert P.H., Use of speech analyses within a mobile application for the assessment of cognitive impairment in elderly people, Curr. Alzheimer Res. 15 (2) (2018) 120–129.

[36]

Low D.M., Bentley K.H., Ghosh S.S., Automated assessment of psychiatric disorders using speech: A systematic review, Laryngoscope Invest. Otolaryngol. 5 (1) (2020) 96–116.

[37]

Ma, X., Yang, H., Chen, Q., Huang, D., Wang, Y., 2016. Depaudionet: An efficient deep model for audio based depression classification. In: Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. pp. 35–42.

[38]

Meilán J.J., Martínez-Sánchez F., Carro J., Sánchez J.A., Pérez E., Acoustic markers associated with impairment in language processing in Alzheimer’s disease, Span. J. Psychol. 15 (2) (2012) 487–494.

[39]

Organization W.H., et al., Global Action Plan on the Public Health Response to Dementia 2017–2025, World Health Organization, 2017.

[40]

Park D.S., Chan W., Zhang Y., Chiu C.-C., Zoph B., Cubuk E.D., Le Q.V., Specaugment: A simple data augmentation method for automatic speech recognition, 2019, arXiv preprint arXiv:1904.08779.

[41]

Petersen R.C., Clinical practice. Mild cognitive impairment, N. Engl. J. Med. 364 (23) (2011) 2227.

[42]

Raju A., Panchapagesan S., Liu X., Mandal A., Strom N., Data augmentation for robust keyword spotting under playback interference, 2018, arXiv preprint arXiv:1808.00563.

[43]

Rogers A., Kovaleva O., Rumshisky A., A primer in bertology: What we know about how bert works, Trans. Assoc. Comput. Linguist. 8 (2021) 842–866.

[44]

Roshanzamir A., Aghajan H., Baghshah M.S., Transformer-based deep neural network language models for Alzheimer’s disease risk assessment from targeted speech, BMC Med. Inf. Decis. Mak. 21 (1) (2021) 1–14.

[45]

Themistocleous C., Eckerström M., Kokkinakis D., Identification of mild cognitive impairment from speech in Swedish using deep sequential neural networks, Front. Neurol. 9 (2018) 975.

[46]

Themistocleous, C., Kokkinakis, D., Eckerström, M., Fraser, K., Fors, K.L., 0000. Effects of Mild Cognitive Impairment on vowel duration.

[47]

Tóth L., Hoffmann I., Gosztolya G., Vincze V., Szatlóczki G., Bánréti Z., Pákáski M., Kálmán J., A speech recognition-based solution for the automatic detection of mild cognitive impairment from spontaneous speech, Curr. Alzheimer Res. 15 (2) (2018) 130–138.

[48]

Vincze V., Gosztolya G., Tóth L., Hoffmann I., Szatlóczki G., Detecting Mild Cognitive Impairment by Exploiting Linguistic Information from Transcripts, Association for Computational Linguistics, 2016.

[49]

Wei Q., Franklin A., Cohen T., Xu H., Clinical text annotation–what factors are associated with the cost of time?, in: AMIA Annual Symposium Proceedings, Vol. 2018, American Medical Informatics Association, 2018, p. 1552.

[50]

Wimo A., Guerchet M., Ali G.-C., Wu Y.-T., Prina A.M., Winblad B., Jönsson L., Liu Z., Prince M., The worldwide costs of dementia 2015 and comparisons with 2010, Alzheimer’s Dement. 13 (1) (2017) 1–7.

[51]

Yu, B., Quatieri, T.F., Williamson, J.R., Mundt, J.C., 2015. Cognitive impairment prediction in the elderly based on vocal biomarkers. In: Sixteenth Annual Conference of the International Speech Communication Association.

Cited By

Pourramezan Fard AMahoor MAlsuhaibani MDodge H(2024)Linguistic-based Mild Cognitive Impairment detection using Informative LossComputers in Biology and Medicine10.1016/j.compbiomed.2024.108606176:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108606
Ilias LAskounis D(2023)Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speechKnowledge-Based Systems10.1016/j.knosys.2023.110834277:COnline publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110834
Ilias LAskounis DPsarras J(2023)Detecting dementia from speech and transcripts using transformersComputer Speech and Language10.1016/j.csl.2023.10148579:COnline publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1016/j.csl.2023.101485
Show More Cited By

Index Terms

An automatic Alzheimer’s disease classifier based on spontaneous spoken English
1. Applied computing
  1. Life and medical sciences
2. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees

Index terms have been assigned to the content through auto-classification.

Recommendations

Diagnosis of Alzheimer's disease using Naive Bayesian Classifier

In the world of modern medicine, though there is lot of medical achievements, some diseases still continue to pest the human race. Unfortunately, dementia is one such disease. All over the world, a large number of people are suffering from dementia. ...
Characterizing and detecting spontaneous speech: Application to speaker role recognition

Processing spontaneous speech is one of the many challenges that automatic speech recognition systems have to deal with. The main characteristics of this kind of speech are disfluencies (filled pause, repetition, false start, etc.) and many studies have ...
Multimodal fusion for alzheimer’s disease recognition
Abstract
Alzheimer’s disease (AD) is the most prevalent form of progressive degenerative dementia, which has a great impact on social economics throughout the world. In the vast majority of cases, AD patients are diagnosed by biochemical analysis, lumbar ...

Comments

Information & Contributors

Information

Published In

cover image Computer Speech and Language

Computer Speech and Language Volume 72, Issue C

Mar 2022

479 pages

ISSN:0885-2308

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Academic Press Ltd.

United Kingdom

Publication History

Published: 01 March 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pourramezan Fard AMahoor MAlsuhaibani MDodge H(2024)Linguistic-based Mild Cognitive Impairment detection using Informative LossComputers in Biology and Medicine10.1016/j.compbiomed.2024.108606176:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108606
Ilias LAskounis D(2023)Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speechKnowledge-Based Systems10.1016/j.knosys.2023.110834277:COnline publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110834
Ilias LAskounis DPsarras J(2023)Detecting dementia from speech and transcripts using transformersComputer Speech and Language10.1016/j.csl.2023.10148579:COnline publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1016/j.csl.2023.101485
Zolnoori MZolnour ATopaz M(2023)ADscreenArtificial Intelligence in Medicine10.1016/j.artmed.2023.102624143:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.artmed.2023.102624
Meng WZhang QMa SCai MLiu DLiu ZYang J(2022)A lightweight CNN and Transformer hybrid model for mental retardation screening among children from spontaneous speechComputers in Biology and Medicine10.1016/j.compbiomed.2022.106281151:PAOnline publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.106281

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents