research-article

Can I Talk to You about Your Social Needs? Understanding Preference for Conversational User Interface in Health

Authors:

Rafal Kocielnik,

Raina Langevin,

James S. George,

Darwin P. Jones,

Alexander Argyle,

Callan Fockele,

Layla Anderson,

Dennis T. Hsieh,

Andrea L. HartzlerAuthors Info & Claims

CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

Article No.: 4, Pages 1 - 10

https://doi.org/10.1145/3469595.3469599

Published: 27 July 2021 Publication History

Abstract

Conversational User Interfaces (CUI) are becoming increasingly utilized in Health applications due to their ability to engage patients and support clinical workflows. Yet recent reviews show that our understanding of CUI performance and user preferences towards them is still lacking. This work examines factors that explain people’s preference for engaging with a social needs screening CUI in a clinical context with 41 emergency department visitors. We demonstrate that people with low health literacy and high attitude towards emotional interaction (AEI) prefer responding to questions via CUI rather than a form-based survey. Specifically, participants with low health literacy appreciate the improved understandability offered by the CUI, whereas participants with high AEI appreciate the added level of engagement offered through conversational interactions. Our work advances the understanding of the benefits of CUI for different user groups in health contexts and beyond.

References

[1]

Viv Bewick, Liz Cheek, and Jonathan Ball. 2005. Statistics review 14: Logistic regression. Critical care 9, 1 (2005), 1–7.

[2]

Timothy Bickmore and Justine Cassell. 2005. Social dialongue with embodied conversational agents. In Advances in natural multimodal dialogue systems. Springer, USA, 23–54.

[3]

Timothy W Bickmore, Laura M Pfeifer, Donna Byron, Shaula Forsythe, Lori E Henault, Brian W Jack, Rebecca Silliman, and Michael K Paasche-Orlow. 2010. Usability of conversational agents by patients with inadequate health literacy: evidence from two clinical trials. Journal of health communication 15, S2 (2010), 197–210.

[4]

Kelly Chandler-Olcott and Donna Mahar. 2003. “Tech-savviness” meets multiliteracies: Exploring adolescent girls’ technology-mediated literacy practices. Reading Research Quarterly 38, 3 (2003), 356–385.

[5]

Ana Paula Chaves and Marco Aurelio Gerosa. 2020. How Should My Chatbot Interact? A Survey on Social Characteristics in Human–Chatbot Interaction Design. International Journal of Human–Computer Interaction 1 (2020), 1–30.

[6]

Leon Ciechanowski, Aleksandra Przegalinska, Mikolaj Magnuski, and Peter Gloor. 2019. In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems 92 (2019), 539–548.

Digital Library

[7]

Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, 2019. What makes a good conversation? Challenges in designing truly conversational agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Montreal, QC, Canada, 1–12.

Digital Library

[8]

Rik Crutzen, Gjalt-Jorn Y Peters, Sarah Dias Portugal, Erwin M Fisser, and Jorne J Grolleman. 2011. An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: an exploratory study. Journal of Adolescent Health 48, 5 (2011), 514–519.

[9]

Terry C Davis, Sandra W Long, Robert H Jackson, EJ Mayeaux, Ronald B George, Peggy W Murphy, and Michael A Crouch. 1993. Rapid estimate of adult literacy in medicine: a shortened screening instrument.Family medicine 25, 6 (1993), 391–395.

[10]

Jasper Feine, Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2019. A taxonomy of social cues for conversational agents. International Journal of Human-Computer Studies 132 (2019), 138–161.

Digital Library

[11]

Sherwin Fernandes, Rutvij Gawas, Preston Alvares, Macklon Femandes, Deepmala Kale, and Shailendra Aswale. 2020. Survey on various conversational systems. In 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE). IEEE, IEEE, USA, 1–8.

[12]

Kathleen Kara Fitzpatrick, Alison Darcy, and Molly Vierhile. 2017. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR mental health 4, 2 (2017), e19.

[13]

Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2018. Faster is Not Always Better: Understanding the Effect of Dynamic Response Delays in Human-Chatbot Interaction. In ECIS. AIS, Germany, 113.

[14]

Ulrich Gnewuch, Stefan Morana, and Alexander Maedche. 2017. Towards Designing Cooperative and Social Conversational Agents for Customer Service. In ICIS. AIS, Germany, 1–10.

[15]

Jonathan Grudin and Richard Jacques. 2019. Chatbots, humbots, and the quest for artificial general intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland, UK, 1–11.

Digital Library

[16]

S Harms, C. & Schmidt. 2017. Conversational Survey Frontends: How ChatbotsCan Improve Online Surveys. https://www.researchgate.net/publication/315199377_Conversational_Survey_Frontends_How_Can_Chatbots_Improve_Online_Survey. [accessed Feb 04 2021].

[17]

Marcel Heerink, Ben Kröse, Vanessa Evers, and Bob Wielinga. 2010. Relating conversational expressiveness to social presence and acceptance of an assistive social robot. Virtual reality 14, 1 (2010), 77–84.

[18]

Thomas Holtgraves and Tai-Lin Han. 2007. A procedure for studying online conversational processing using a chat bot. Behavior research methods 39, 1 (2007), 156–163.

[19]

C Hong, D Hsieh, P Liu, and S Johnson. 2018. ”White paper: Development of a social and behavioral determinants of health short screener.”. UCLA County 1, 1 (2018), 1–10.

[20]

Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N Patel. 2018. Evaluating and informing the design of chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, Hong Kong, China, 895–906.

Digital Library

[21]

Liyin Jin. 2011. Improving response rates in web surveys with default setting: The effects of default on web survey participation and permission. International Journal of Market Research 53, 1 (2011), 75–94.

[22]

Yuta Katsumi, Suhkyung Kim, Keen Sung, Florin Dolcos, and Sanda Dolcos. 2017. When nonverbal greetings “make it or break it”: the role of ethnicity and gender in the effect of handshake on social appraisals. Journal of Nonverbal Behavior 41, 4 (2017), 345–365.

[23]

Soomin Kim, Joonhwan Lee, and Gahgene Gweon. 2019. Comparing data from chatbot and web surveys: Effects of platform and conversational style on survey response quality. In Proceedings of the 2019 CHI conference on human factors in computing systems. ACM, Glasgow Scotland, UK, 1–12.

Digital Library

[24]

Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The rise of bots: A survey of conversational interfaces, patterns, and paradigms. In Proceedings of the 2017 conference on designing interactive systems. ACM, Edinburgh, UK, 555–565.

Digital Library

[25]

A Baki Kocaballi, Juan C Quiroz, Liliana Laranjo, Dana Rezazadegan, Rafal Kocielnik, Leigh Clark, Q Vera Liao, Sun Young Park, Robert J Moore, and Adam Miner. 2020. Conversational Agents for Health and Wellbeing. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–8.

[26]

Rafal Kocielnik, Elena Agapie, Alexander Argyle, Dennis T Hsieh, Kabir Yadav, Breena Taira, and Gary Hsieh. 2019. HarborBot: A Chatbot for Social Needs Screening. In AMIA Annual Symposium Proceedings, Vol. 2019. American Medical Informatics Association, American Medical Informatics Association, Washington, USA, 552.

[27]

Rafal Kocielnik, Saleema Amershi, and Paul N Bennett. 2019. Will you accept an imperfect ai? exploring designs for adjusting end-user expectations of ai systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland, UK, 1–14.

Digital Library

[28]

Mark Kutner, Elizabeth Greenburg, Ying Jin, and Christine Paulsen. 2006. The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. NCES 2006-483.National Center for Education Statistics 1, 1 (2006), 1–10.

[29]

Liliana Laranjo, Adam G Dunn, Huong Ly Tong, Ahmet Baki Kocaballi, Jessica Chen, Rabia Bashir, Didi Surian, Blanca Gallego, Farah Magrabi, Annie YS Lau, 2018. Conversational agents in healthcare: a systematic review. Journal of the American Medical Informatics Association 25, 9(2018), 1248–1258.

[30]

Q Vera Liao, Matthew Davis, Werner Geyer, Michael Muller, and N Sadat Shami. 2016. What can you do? Studying social-agent orientation and agent proactive interactions with an agent for employees. In Proceedings of the 2016 acm conference on designing interactive systems. ACM, Brisbane QLD Australia, 264–275.

Digital Library

[31]

Q Vera Liao, Muhammed Mas-ud Hussain, Praveen Chandar, Matthew Davis, Yasaman Khazaeni, Marco Patricio Crasso, Dakuo Wang, Michael Muller, N Sadat Shami, and Werner Geyer. 2018. All work and no play?. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal, QC, Canada, 1–13.

Digital Library

[32]

G. Lucas, J. Boberg, D. Traum, Ron Artstein, J. Gratch, Alesia Gainer, E. Johnson, A. Leuski, and M. Nakano. 2017. The Role of Social Dialogue and Errors in Robots. Proceedings of the 5th International Conference on Human Agent Interaction 5, 1 (2017), 431–433.

Digital Library

[33]

Ewa Luger and Abigail Sellen. 2016. ” Like Having a Really Bad PA” The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI conference on human factors in computing systems. ACM, San Jose, USA, 5286–5297.

Digital Library

[34]

David S Metzger, Beryl Koblin, Charles Turner, Helen Navaline, Francesca Valenti, Sarah Holte, Michael Gross, Amy Sheon, Heather Miller, Philip Cooley, 2000. Randomized controlled trial of audio computer-assisted self-interviewing: utility and acceptability in longitudinal studies. American journal of epidemiology 152, 2 (2000), 99–106.

[35]

Joao Luis Zeni Montenegro, Cristiano André da Costa, and Rodrigo da Rosa Righi. 2019. Survey of conversational agents in health. Expert Systems with Applications 129 (2019), 56–67.

Digital Library

[36]

Tatsuya Nomura, Takayuki Kanda, Tomohiro Suzuki, and Kensuke Kato. 2008. Prediction of human behavior in human–robot interaction using psychological scales for anxiety and negative attitudes toward robots. IEEE transactions on robotics 24, 2 (2008), 442–451.

Digital Library

[37]

Heather L O’Brien and Elaine G Toms. 2010. The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology 61, 1 (2010), 50–69.

[38]

The University of Chicago Student Healthand Counseling Services. 2018. Speed Reading. https://web.archive.org/web/20180307083117/https://wellness.uchicago.edu/page/speed-reading/. [Online; Archived from the original on 7 March 2018. Retrieved 30 December 2017.].

[39]

Manuel Portela and Carlos Granell-Canut. 2017. A new friend in our Smartphone? Observing Interactions with Chatbots in the search of emotional engagement. In Proceedings of the XVIII International Conference on Human Computer Interaction. ACM, Cancun Mexico, 1–7.

Digital Library

[40]

Susan Robinson, David R Traum, Midhun Ittycheriah, and Joe Henderer. 2008. What would you Ask a conversational Agent? Observations of Human-Agent Dialogues in a Museum Setting. In LREC. European Language Resources Association (ELRA), Marrakech, Morocco, 1–11.

[41]

Susana Rubio, Eva Díaz, Jesús Martín, and José M Puente. 2004. Evaluation of subjective mental workload: A comparison of SWAT, NASA-TLX, and workload profile methods. Applied psychology 53, 1 (2004), 61–86.

[42]

Shalom H Schwartz. 2012. An overview of the Schwartz theory of basic values. Online readings in Psychology and Culture 2, 1 (2012), 2307–0919.

[43]

Nina Svenningsson and Montathar Faraon. 2019. Artificial intelligence in conversational agents: A study of factors related to perceived humanness in chatbots. In Proceedings of the 2019 2nd Artificial Intelligence and Cloud Computing Conference. ACM, USA, 151–161.

Digital Library

[44]

Catharine Wang, Timothy Bickmore, Deborah J Bowen, Tricia Norkunas, MaryAnn Campion, Howard Cabral, Michael Winter, and Michael Paasche-Orlow. 2015. Acceptability and feasibility of a virtual counselor (VICKY) to collect family health histories. Genetics in Medicine 17, 10 (2015), 822–830.

[45]

Barry D Weiss, Mary Z Mays, William Martz, Kelley Merriam Castro, Darren A DeWalt, Michael P Pignone, Joy Mockbee, and Frank A Hale. 2005. Quick assessment of literacy in primary care: the newest vital sign. The Annals of Family Medicine 3, 6 (2005), 514–522.

[46]

Ziang Xiao, Michelle X Zhou, Q Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, and Huahai Yang. 2020. Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. ACM Transactions on Computer-Human Interaction (TOCHI) 27, 3(2020), 1–37.

Digital Library

[47]

Zhaopeng Xing, Fei Yu, Jian Du, Jennifer S Walker, Claire B Paulson, Nandita S Mani, and Lixin Song. 2019. Conversational interfaces for health: bibliometric analysis of grants, publications, and patents. Journal of medical Internet research 21, 11 (2019), e14672.

[48]

Lisa M Pytlik Zillig, Scott H Hemenover, and Richard A Dienstbier. 2002. What do we assess when we assess a Big 5 trait? A content analysis of the affective, behavioral, and cognitive processes represented in Big 5 personality inventories. Personality and Social Psychology Bulletin 28, 6 (2002), 847–858.

Cited By

Sezgin EKocaballi ADolce MSkeens MMilitello LHuang YStevens JKemper A(2024)Chatbot for Social Need Screening and Resource Sharing With Vulnerable Families: Iterative Design and Evaluation StudyJMIR Human Factors10.2196/5711411(e57114)Online publication date: 19-Jul-2024
https://doi.org/10.2196/57114
Cuadra ABreuch JEstrada SIhim DHung IAskaryar DHassanien MFessele KLanday J(2024)Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data EntryProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596248:2(1-39)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659624
Alberts LLyngs UVan Kleek M(2024)Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act SociallyProceedings of the ACM on Human-Computer Interaction10.1145/36536938:CSCW1(1-25)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3653693
Show More Cited By

Recommendations

Understanding is a Two-Way Street: User-Initiated Repair on Agent Responses and Hearing in Conversational Interfaces
CSCW

Although methods for repairing prior turns in natural conversation are critical for enabling mutual understanding, or successful communication, these methods are seldom built into conversational user interfaces systematically. Chatbots and voice ...
Conversational Agents: Acting on the Wave of Research and Development
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

In the last five years, work on software that interacts with people via typed or spoken natural language, called chatbots, intelligent assistants, social bots, virtual companions, non-human players, and so on, increased dramatically. Chatbots burst into ...
The Effect of Emojis when interacting with Conversational Interface Assisted Health Coaching System
PervasiveHealth '18: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare

The recent rise of conversational interfaces have made it possible to integrate this technology into various domains, among which is health. Dialogue systems and conversational agents can bring a lot into healthcare to reduce cost, increase efficiency ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

July 2021

262 pages

ISBN:9781450389983

DOI:10.1145/3469595

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Institutes of Health

Conference

CUI '21

CUI '21: CUI 2021 - 3rd Conference on Conversational User Interfaces

July 27 - 29, 2021

Bilbao (online), Spain

Acceptance Rates

Overall Acceptance Rate 34 of 100 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
550
Total Downloads

Downloads (Last 12 months)138
Downloads (Last 6 weeks)16

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sezgin EKocaballi ADolce MSkeens MMilitello LHuang YStevens JKemper A(2024)Chatbot for Social Need Screening and Resource Sharing With Vulnerable Families: Iterative Design and Evaluation StudyJMIR Human Factors10.2196/5711411(e57114)Online publication date: 19-Jul-2024
https://doi.org/10.2196/57114
Cuadra ABreuch JEstrada SIhim DHung IAskaryar DHassanien MFessele KLanday J(2024)Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data EntryProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596248:2(1-39)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659624
Alberts LLyngs UVan Kleek M(2024)Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act SociallyProceedings of the ACM on Human-Computer Interaction10.1145/36536938:CSCW1(1-25)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3653693
Rey BSakamoto YSin JIrani P(2024)Understanding User Preferences of Voice Assistant Answer Structures for Personal Health Data QueriesProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665552(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665552
Desai SDubiel MLeiva L(2024)Examining Humanness as a Metaphor to Design Voice User InterfacesProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665535(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665535
Alsharhan AAl-Emran MShaalan K(2024)Chatbot Adoption: A Multiperspective Systematic Review and Future Research AgendaIEEE Transactions on Engineering Management10.1109/TEM.2023.329836071(10232-10244)Online publication date: 2024
https://doi.org/10.1109/TEM.2023.3298360
Dubiel MBarghouti YKudryavtseva KLeiva L(2024)On-device query intent prediction with lightweight LLMs to support ubiquitous conversationsScientific Reports10.1038/s41598-024-63380-614:1Online publication date: 3-Jun-2024
https://doi.org/10.1038/s41598-024-63380-6
Wester JLee Mvan Berkel N(2023)Moral Transparency as a Mitigator of Moral Bias in Conversational User InterfacesProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603752(1-6)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3603752
Houben Mvan As NSawhney NUnbehaun DLee M(2023)Participatory Design for Whom? Designing Conversational User Interfaces for Sensitive Settings and Vulnerable PopulationsProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597439(1-4)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3597439
Lupetti MHagens EVan Der Maden WSteegers-Theunissen RRousian M(2023)Trustworthy Embodied Conversational Agents for HealthcareProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597128(1-14)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3597128
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents