Conversational Assessment of Mild Cognitive Impairment with Virtual Agents
Authors: 
Emily E. Hurstak, 
Stefan Olafsson, Teresa K. O'Leary, Howard J. Cabral, Michael Paasche-Orlow, 
Timothy BickmoreAuthors Info & Claims
Emily E. Hurstak, Stefan Olafsson, Teresa K. O'Leary, Howard J. Cabral, Michael Paasche-Orlow, Timothy BickmoreAuthors Info & ClaimsArticle No.: 8, Pages 1 - 8
Abstract
Over 55 million adults worldwide have dementia, a syndrome characterized by deterioration in cognitive functioning. Screening for mild cognitive impairment is important to identify dementia early to facilitate diagnosis and initiate treatment that may modify the disease trajectory. However, standard cognitive screening tools are time-consuming, require expert administration, and make people feel as if they are being tested and are thus potentially stigmatizing. Consequently, standard cognitive screening tools are underutilized, and dementia is often identified much later in the disease process leading to greater health-related and societal harms. We explored cognitive ability assessments using virtual agents, in which assessments are made during conversational dialogues. We evaluated four strategies for the assessment of cognitive ability in a sample of 41 individuals with varying cognitive ability, comparing each strategy to scores on the Mini Mental Status Exam (MMSE), the gold standard clinical screening tool for cognitive impairment. One of the four conversational strategies demonstrated significant agreement with the MMSE, with a sensitivity of 92.3% and a specificity of 48.3% for identification of impairment compared to the MMSE. Participants reported moderate levels of satisfaction with the agent-based conversational assessment. This approach could be used to identify individuals warranting more evaluation for possible cognitive impairment.
References
[1]
World Health Organization Risk reduction of cognitive decline and dementia: WHO guidelines (2019).
[2]
Gauthier, S., Reisberg, B., Zaudig, M., Petersen, R. C., Ritchie, K., Broich, K., Belleville, S., Brodaty, H., Bennett, D. and Chertkow, H. Mild cognitive impairment. The lancet, 367, 9518 (2006), 1262--1270.
[3]
Ward, A., Arrighi, H. M., Michels, S. and Cedarbaum, J. M. Mild cognitive impairment: disparity of incidence and prevalence estimates. Alzheimers Dement, 8, 1 (Jan 2012), 14--21.
[4]
Petersen, R. C., Lopez, O., Armstrong, M. J., Getchius, T. S. D., Ganguli, M., Gloss, D., Gronseth, G. S., Marson, D., Pringsheim, T., Day, G. S., Sager, M., Stevens, J. and Rae-Grant, A. Practice guideline update summary: Mild cognitive impairment: Report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology. Neurology, 90, 3 (Jan 16 2018), 126--135.
[5]
Roberts, R. and Knopman, D. S. Classification and epidemiology of MCI. Clinics in geriatric medicine, 29, 4 (Nov 2013), 753--772.
[6]
Langa, K. M. and Levine, D. A. The diagnosis and management of mild cognitive impairment: a clinical review. Jama, 312, 23 (Dec 17 2014), 2551--2561.
[7]
Patnode, C. D., Perdue, L. A., Rossom, R. C., Rushkin, M. C., Redmond, N., Thomas, R. G. and Lin, J. S. Screening for cognitive impairment in older adults: updated evidence report and systematic review for the US Preventive Services Task Force. Jama, 323, 8 (2020), 764--785.
[8]
Frank, L., Lloyd, A., Flynn, J. A., Kleinman, L., Matza, L. S., Margolis, M. K., Bowman, L. and Bullock, R. Impact of cognitive impairment on mild dementia patients and mild cognitive impairment patients and their informants. International Psychogeriatrics, 18, 1 (2006), 151--162.
[9]
Bradford, A., Kunik, M. E., Schulz, P., Williams, S. P. and Singh, H. Missed and delayed diagnosis of dementia in primary care: prevalence and contributing factors. Alzheimer Disease & Associated Disorders, 23, 4 (2009), 306--314. https://doi.org/10.1097/WAD. 0b013e3181a6bebc.
[10]
Sabbagh, M. N., Boada, M., Borson, S., Chilukuri, M., Dubois, B., Ingram, J., Iwata, A., Porsteinsson, A. P., Possin, K. L., Rabinovici, G. D., Vellas, B., Chao, S., Vergallo, A. and Hampel, H. Early Detection of Mild Cognitive Impairment (MCI) in Primary Care. J Prev Alzheimers Dis, 7, 3 (2020), 165--170.
[11]
Bernstein, A., Rogers, K. M., Possin, K. L., Steele, N. Z., Ritchie, C. S., Miller, B. L. and Rankin, K. P. Primary care provider attitudes and practices evaluating and managing patients with neurocognitive disorders. Journal of General Internal Medicine, 34 (2019), 1691--1692.
[12]
Sidner, C. L., Rich, C., Shayganfar, M., Bickmore, T. W., Ring, L. and Zhang, Z. A Robotic Companion for Social Support of Isolated Older Adults. HRI (Extended Abstracts), 289, 10.1145 (2015), 2701973--2702103.
[13]
Vardoulakis, L. P., Ring, L., Barry, B., Sidner, C. L. and Bickmore, T. Designing relational agents as long term social companions for older adults. In Intelligent Virtual Agents, 12th International Conference, September 12-14, 2012. Santa Cruz, CA, USA, Springer Berlin Heidelberg, 289--302.
[14]
Yaghoubzadeh, R., Kramer, M., Pitsch, K. and Kopp, S. Virtual agents as daily assistants for elderly or cognitively impaired people: Studies on acceptance and interaction feasibility. In Intelligent Virtual Agents, 13th International Conference, Edinburgh, UK, August 29-31, 2013. Springer Berlin Heidelberg, 79--91.
[15]
Van Welbergen, H., Yaghoubzadeh, R. and Kopp, S. AsapRealizer 2.0: The next steps in fluent behavior realization for ECAs. In Intelligent Virtual Agents: 14th International Conference, Boston, MA, USA, August 27-29, 2014. Springer International Publishing, 449--462.
[16]
Bickmore, T. W., Silliman, R. A., Nelson, K., Cheng, D. M., Winter, M., Henault, L. and Paasche-Orlow, M. K. A randomized controlled trial of an automated exercise coach for older adults. Journal of the American Geriatrics Society, 61, 10 (2013), 1676--1683. 10.1111/jgs.12449.
[17]
Eschweiler, G. and Wanner, L. How Can Intelligent Conversational Agents Help? The Needs of Geriatric Patients and Their Caregivers. In Proceedings of the ICAHGCA@AAMAS, Stockholm, Sweden, 15 July 2018, 1--9.
[18]
El Kamali, M., Angelini, L., Caon, M., Andreoni, G., Khaled, O. A. and Mugellini, E. Towards the NESTORE e-Coach: a tangible and embodied conversational agent for older adults. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, 1656--1663.
[19]
Sidner, C. L., Bickmore, T. W., Rich, C., Barry, B., Ring, L., Behrooz, M. and Shayganfar, M. Demonstration of an always-on companion for isolated older adults. In Proceedings of the SIGDIAL 2013 Conference, August 22-24, 2013, Metz, France, 148--150.
[20]
Auriacombe, M., Moriceau, S., Serre, F., Denis, C., Micoulaud-Franchi, J.-A., de Sevin, E., Bonhomme, E., Bioulac, S., Fatseas, M. and Philip, P. Development and validation of a virtual agent to screen tobacco and alcohol use disorders. Drug and alcohol dependence, 193 (2018), 1--6.
[21]
Martínez-Miranda, J., Martínez, A., Ramos, R., Aguilar, H., Jiménez, L., Arias, H., Rosales, G. and Valencia, E. Assessment of users' acceptability of a mobile-based embodied conversational agent for the prevention and detection of suicidal behaviour. Journal of medical systems, 43, 8 (2019), 246. 0.1007/s10916-019-1387-1.
[22]
Philip, P., Micoulaud-Franchi, J.-A., Sagaspe, P., Sevin, E. D., Olive, J., Bioulac, S. and Sauteraud, A. Virtual human as a new diagnostic tool, a proof of concept study in the field of major depressive disorders. Scientific reports, 7, 1 (2017), 1--7.
[23]
Simon, S. R., Checchi, K., McNair, S. S., Rubin, A., Marcello, T. and Bickmore, T. A PILOT STUDY OF A COMPUTER-BASED RELATIONAL AGENT TO SCREEN FOR SUBSTANCE-USE PROBLEMS IN PRIIMARY CARE. In Society of General Internal Medicine Annual Meeting, Denver, Colorado April 24-27, 2013. Springer, New York, NY, 2013.
[24]
Provoost, S., Lau, H. M., Ruwaard, J. and Riper, H. Embodied conversational agents in clinical psychology: a scoping review. Journal of medical Internet research, 19, 5 (2017), e151.
[25]
Pontier, M. and Siddiqui, G. F. A virtual therapist that responds empathically to your answers. In Intelligent Virtual Agents: 8th International Conference, Tokyo, Japan, September 1-3, 2008. Proceedings 8, Springer, Berlin, Heidelberg, 417--425.
[26]
Jaiswal, S., Valstar, M., Kusumam, K. and Greenhalgh, C. Virtual human questionnaire for analysis of depression, anxiety and personality. In Proceedings of the 19th ACM international conference on intelligent virtual agents. Association for Computing Machinery, Inc (2019), 81--87.
[27]
Bickmore, T., Rubin, A. and Simon, S. Substance Use Screening using Virtual Agents: Towards Automated Screening, Brief Intervention, and Referral to Treatment (SBIRT). In Proceedings of the ACM International Conference on Intelligent Virtual Agents (IVA), Glasgow; Virtual Event Scotland UK, ACM, 2020, 1--7.
[28]
Roark, B., Mitchell, M., Hosom, J.-P., Hollingshead, K. and Kaye, J. Spoken language derived measures for detecting mild cognitive impairment. IEEE transactions on audio, speech, and language processing, 19, 7 (2011), 2081--2090.
[29]
Aramaki, E., Shikata, S., Miyabe, M. and Kinoshita, A. Vocabulary size in speech may be an early indicator of cognitive impairment. PloS one, 11, 5 (2016), e0155195.
[30]
Tang, F., Uchendu, I., Wang, F., Dodge, H. H. and Zhou, J. Scalable diagnostic screening of mild cognitive impairment using AI dialogue agent. Scientific reports, 10, 1 (2020/03/31 2020), 5732.
[31]
Tanaka, H., Adachi, H., Ukita, N., Ikeda, M., Kazui, H., Kudo, T. and Nakamura, S. Detecting dementia through interactive computer avatars. IEEE journal of translational engineering in health and medicine, 5 (2017), 1--11.
[32]
Ujiro, T., Tanaka, H., Adachi, H., Kazui, H., Ikeda, M., Kudo, T. and Nakamura, S. Detection of Dementia from Responses to Atypical Questions Asked by Embodied Conversational Agents. In Interspeech 2018, 1691--1695.
[33]
Mirheidari, B., Blackburn, D., O'Malley, R., Walker, T., Venneri, A., Reuber, M. and Christensen, H. Computational cognitive assessment: Investigating the use of an intelligent virtual agent for the detection of early signs of dementia. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 12, 2019, IEEE, 2732--2736.
[34]
Kobayashi, M., Kosugi, A., Takagi, H., Nemoto, M., Nemoto, K., Arai, T. and Yamada, Y. Effects of age-related cognitive decline on elderly user interactions with voice-based dialogue systems. In Human-Computer Interaction--INTERACT 2019: 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2-6, 2019, Springer International Publishing, 53--74.
[35]
Cassell, J., Vilhjálmsson, H. and Bickmore, T. BEAT: The Behavior Expression Animation Toolkit. In Proceedings of the SIGGRAPH '01, August 2001, Los Angeles, CA, ACM Press, New York, New York, 477--486.
[36]
Iverson, G. L., Marsh, J. M., Connors, E. J. and Terry, D. P. Normative reference values, reliability, and item-level symptom endorsement for the PROMIS® v2. 0 cognitive function-short forms 4a, 6a and 8a. Archives of clinical neuropsychology, 36, 7 (2021), 1341--1349.
[37]
Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J. L. and Chertkow, H. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53, 4 (2005), 695--699.
[38]
Storey, J. E., Rowland, J. T., Conforti, D. A. and Dickson, H. G. The Rowland universal dementia assessment scale (RUDAS): a multicultural cognitive assessment scale. International Psychogeriatrics, 16, 1 (2004), 13--31.
[39]
Folstein, M. F., Robins, L. N. and Helzer, J. E. The mini-mental state examination. Arch. Gen. Psychiatry, 40, 7 (1983), 812--812.
[40]
Munder, T., Wilmers, F., Leonhart, R., Linster, H. W. and Barth, J. Working Alliance Inventory-Short Revised (WAI-SR): psychometric properties in outpatients and inpatients. Clin Psychol Psychother, 17, 3 (May-Jun 2010), 231--239.
[41]
Norris, D., Clark, M. S. and Shipley, S. The Mental Status Examination. American family physician, 94, 8 (Oct 15 2016), 635--641.
[42]
Murali, P., Trinh, H., Ring, L. and Bickmore, T. A friendly face in the crowd: Reducing public speaking anxiety with an emotional support agent in the audience. In Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents, Virtual Event Japan, September 14-17, 2021, ACM, New York, NY, 156--163.
[43]
Taylor, B., Henshall, C., Kenyon, S., Litchfield, I. and Greenfield, S. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ open, 8, 10 (2018), e019993.
Index Terms
- Conversational Assessment of Mild Cognitive Impairment with Virtual Agents
Recommendations
Towards Automated Pain Assessment using Embodied Conversational Agents
ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal InteractionNarrative accounts are the ultimate authoritative source for pain assessment, and face-to-face encounters provide a rich context in which nonverbal conversational behavior can be used to enrich the detail in these descriptions. Embodied Conversational ...
Modelling cognitive impairment to improve universal access
UAHCI'11: Proceedings of the 6th international conference on Universal access in human-computer interaction: users diversity - Volume Part IIThe purpose of this study is to develop a model of cognitive impairment to help designers consider the range of issues which affect the lives of people living with such impairment. A series of interviews with experts of cognitive impairment was ...
Engaging with an embodied conversational agent in a computerized cognitive training: an acceptability study with the elderly
ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal InteractionNeurodegenerative diseases have become increasingly prevalent, particularly among the elderly population, with conditions such as Alzheimer's disease, Parkinson's disease, and Mild Cognitive Impairment (MCI) affecting millions of individuals in France. ...
Comments
Information & Contributors
Information
Published In
September 2023
376 pages
ISBN:9781450399944
DOI:10.1145/3570945
- General Chairs:
- Birgit Lugrin,
- Marc Latoschik,
- Sebastion von Mammen,
- Program Chairs:
- Stefan Kopp,
- Florian Pécune,
- Catherine Pelachaud
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
IVA '23
Sponsor:
IVA '23: ACM International Conference on Intelligent Virtual Agents
September 19 - 22, 2023
Würzburg, Germany
Acceptance Rates
Overall Acceptance Rate 53 of 196 submissions, 27%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 114Total Downloads
- Downloads (Last 12 months)114
- Downloads (Last 6 weeks)10
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in