short-paper

Spill the Tea: When Robot Conversation Agents Support Well-being for Older Adults

Authors:

Kyrie Jig Amon,

Pranav Vijay Chakilam,

Zachary Kaufman,

Manasi Swaminathan,

Katherine M. Tsui,

David J. Crandall,

Selma SabanovićAuthors Info & Claims

HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

Pages 178 - 182

https://doi.org/10.1145/3568294.3580067

Published: 13 March 2023 Publication History

Abstract

Robots could support older adults' well-being by engaging them in meaningful conversations, specifically to reflect on, support, and improve different aspects of their well-being. We implemented a system on a QT social robot to conduct short autonomous conversations with older adults, to help understand what brings them feelings of joy and meaning in life. We evaluated the system with written surveys and observations of 12 participants including older adults, caregivers, and dementia care staff. From this, we saw the need to improve user experience through personalized interaction that better support older adults as they talk about well-being. Improving the interactions will involve improving the conversation flow, detecting emotions and nonverbal cues, and natural language processing to extract topics around well-being.

References

[1]

Shu-Chuan Chen, Cindy Jones, and Wendy Moyle. 2018. Social robots for depression in older adults: a systematic review. Journal of Nursing Scholarship, Vol. 50, 6 (2018), 612--622.

[2]

Eunji Chong, Elysha Clark-Whitney, Audrey Southerland, Elizabeth Stubbs, Chanel Miller, Eliana L Ajodan, Melanie R Silverman, Catherine Lord, Agata Rozga, Rebecca M Jones, et al. 2020. Detection of eye contact with deep neural networks is as accurate as human experts. Nature communications, Vol. 11, 1 (2020), 1--10.

[3]

genderize.io. Last Accessed December 7, 2022. Genderize Our Data. [Online]. Available from: https://genderize.io/our-data.

[4]

Google. Last Accessed December 7, 2022. MediaPipe Face Detection. [Online]. Available from: https://google.github.io/mediapipe/solutions/face_detection.

[5]

Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear (2017).

[6]

Kun-Yi Huang, Chung-Hsien Wu, Qian-Bei Hong, Ming-Hsiang Su, and Yi-Hsuan Chen. 2019. Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds. 5866--5870. https://doi.org/10.1109/ICASSP.2019.8682283

[7]

Suzanne Hutson, Soo Ling Lim, Peter J Bentley, Nadia Bianchi-Berthouze, and Ann Bowling. 2011. Investigating the suitability of social robots for the wellbeing of the elderly. In International conference on affective computing and intelligent interaction. Springer, 578--587.

[8]

IBM. Last Accessed December 7, 2022. IBM Watson Natural Language Understanding Entity Types. [Online]. Available from: https://cloud.ibm.com/docs/natural-language-understanding.

[9]

Becky Inkster, Shubhankar Sarda, Vinod Subramanian, et al. 2018. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR mHealth and uHealth, Vol. 6, 11 (2018), e12106.

[10]

LuxAI. Last Accessed December 7, 2022. QTrobot V2 Humanoid Social Robot for Human AI Research & Teaching. https://luxai.com/qtrobot-for-research/

[11]

Raju Maharjan, Kevin Doherty, Darius Adam Rohani, Per Bækgaard, and Jakob E Bardram. 2022. Experiences of a Speech-enabled Conversational Agent for the Self-report of Well-being among People Living with Affective Disorders: An In-the-Wild Study. ACM Transactions on Interactive Intelligent Systems (TiiS), Vol. 12, 2 (2022), 1--29.

Digital Library

[12]

Bernt Meerbeek, Martin Saerbeck, and Christoph Bartneck. 2009. Iterative design process for robots with personality. In AISB2009 Symposium on new frontiers in human-robot interaction. 94--101.

[13]

OpenAI. Last Accessed December 7, 2022. Open AI Models. [Online]. Available from: https://beta.openai.com/docs/models/gpt-3.

[14]

Lihui Pu, Wendy Moyle, Cindy Jones, and Michael Todorovic. 2019. The effectiveness of social robots for older adults: a systematic review and meta-analysis of randomized controlled studies. The Gerontologist, Vol. 59, 1 (2019), e37--e51.

[15]

Natasha Randall, Swapna Joshi, Waki Kamino, Long-Jing Hsu, Abhijeet Agnihotri, Grace Li, Donald Williamson, Kate Tsui, and Selma vS abanović. 2022. Finding ikigai: How robots can support meaning in later life. Frontiers in Robotics and AI, Vol. 9 (2022).

[16]

Korok Sengupta, Sayan Sarcar, Alisha Pradhan, Roisin McNaney, Sergio Sayago, Debaleena Chattopadhyay, and Anirudha Joshi. 2020. Challenges and Opportunities of Leveraging Intelligent Conversational Assistant to Improve the Well-Being of Older Adults. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--4. https://doi.org/10.1145/3334480.3381057

Digital Library

[17]

Gabriel Skantze. 2021. Turn-taking in Conversational Systems and Human-Robot Interaction: A Review. Computer Speech & Language, Vol. 67 (2021), 101178. https://doi.org/10.1016/j.csl.2020.101178

[18]

Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, and Haizhou Li. 2021. Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection. In Proceedings of the 29th ACM International Conference on Multimedia. 3927--3935.

Digital Library

[19]

Antoine Toisoul, Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, and Maja Pantic. 2021a. Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Machine Intelligence (2021). https://www.nature.com/articles/s42256-020-00280-0

[20]

Antoine Toisoul, Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, and Maja pantic. 2021b. Face-analysis/emonet: Official implementation of the paper "estimation of continuous valence and arousal levels from faces in naturalistic conditions. [Online]. Available from: https://github.com/face-analysis/emonet.

[21]

Anthony Zhang. Last Accessed December 7, 2022. SpeechRecognition 3.9.0. [Online]. Available from: https://pypi.org/project/SpeechRecognition/.

Cited By

Hsu LKhoo WGoshomi PStafford PSwaminathan MTsui KCrandall DSabanović SGrollman DBroadbent EJu WSoh HWilliams T(2024)Is Now a Good Time? Opportune Moments for Interacting with an Ikigai Support RobotCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640666(549-553)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640666
Swaminathan MHsu LThant MAmon KKim ATsui KSabanović SCrandall DKhoo WGrollman DBroadbent EJu WSoh HWilliams T(2024)If [YourName] Can Code, So Can You! End-User Robot Programming For Non-ExpertsCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640644(1033-1037)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640644
Hsu LStafford PKhoo WSwaminathan MAmon KSato HTsui KCrandall DSabanović SGrollman DBroadbent EJu WSoh HWilliams T(2024)"Give it Time:" Longitudinal Panels Scaffold Older Adults' Learning and Robot Co-DesignProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634979(283-292)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634979
Show More Cited By

Index Terms

Spill the Tea: When Robot Conversation Agents Support Well-being for Older Adults

Recommendations

If [YourName] Can Code, So Can You! End-User Robot Programming For Non-Experts
HRI '24: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

Social robots are being studied for a wide variety of user populations, such as older adults, but programming these social robots typically requires deep technical knowledge. In this study, we developed a no-code end-user robot programming interface, ...
Interaction design for robotic avatars does avatar's aging cue affect the user's impressions of a robot?
UAHCI'13: Proceedings of the 7th international conference on Universal Access in Human-Computer Interaction: user and context diversity - Volume 2

Human Computer/Robot Interaction has concerned about developing embodied computer/robot agents effective for their use in user interaction. In particular, the study on the dependency of the interaction design on the target users has been of a core theme ...
Assistive Technologies for Supporting Wellbeing of Older Adults
PETRA '21: Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference

Over the course of the coming decades, the societal goal of maximizing the number of older adults who experience positive trajectories of ageing has been established. To this end, innovative technology-based interventions tailored to older adults’ ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

March 2023

612 pages

ISBN:9781450399708

DOI:10.1145/3568294

General Chairs:
Ginevra Castellano
Uppsala University, Sweden
,
Laurel Riek
University of California San Diego, USA
,
Program Chairs:
Maya Cakmak
University of Washington, USA
,
Iolanda Leite
KTH Royal Institute of Technology, Sweden

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Toyota Research Institute

Conference

HRI '23

Sponsor:

HRI '23: ACM/IEEE International Conference on Human-Robot Interaction

March 13 - 16, 2023

Stockholm, Sweden

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
306
Total Downloads

Downloads (Last 12 months)190
Downloads (Last 6 weeks)25

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hsu LKhoo WGoshomi PStafford PSwaminathan MTsui KCrandall DSabanović SGrollman DBroadbent EJu WSoh HWilliams T(2024)Is Now a Good Time? Opportune Moments for Interacting with an Ikigai Support RobotCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640666(549-553)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640666
Swaminathan MHsu LThant MAmon KKim ATsui KSabanović SCrandall DKhoo WGrollman DBroadbent EJu WSoh HWilliams T(2024)If [YourName] Can Code, So Can You! End-User Robot Programming For Non-ExpertsCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640644(1033-1037)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640644
Hsu LStafford PKhoo WSwaminathan MAmon KSato HTsui KCrandall DSabanović SGrollman DBroadbent EJu WSoh HWilliams T(2024)"Give it Time:" Longitudinal Panels Scaffold Older Adults' Learning and Robot Co-DesignProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634979(283-292)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634979
Kim CLee CMutlu BGrollman DBroadbent EJu WSoh HWilliams T(2024)Understanding Large-Language Model (LLM)-powered Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634966(371-380)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634966
Kim YKim DChoi JPark JOh NPark D(2024)A survey on integration of large language models with intelligent robotsIntelligent Service Robotics10.1007/s11370-024-00550-517:5(1091-1107)Online publication date: 13-Aug-2024
https://doi.org/10.1007/s11370-024-00550-5
Hsu LBays JTsui KSabanovic S(2023)Co-designing Social Robots with People Living with Dementia: Fostering Identity, Connectedness, Security, and AutonomyProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595987(2672-2688)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595987
Hsu LKhoo WRandall NKamino WJoshi SSato HCrandall DTsui KŠabanović S(2023)Finding its Voice: The Influence of Robot Voice on Fit, Social Attributes, and Willingness to Use Among Older Adults in the U.S. and Japan2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309390(2072-2079)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309390

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents