Exploring the Impact of User-Participated Customization in Experiencing Chatbot Failure
Article No.: 153, Pages 1 - 7
Abstract
Traditionally, personalized AI systems have offered customized experiences by inferring user preferences from their system usage data, which can be referred to as data-driven personalization. With the advent of GPTs, opportunities for users to directly engage in the customization process have increased, which we term user-participated customization. In this study, we explored the impact of user-participated customization on user reactions to chatbot failures. We involved twenty-two participants in total, with fourteen for user-participated customization and eight for data-driven personalization. Both groups experienced frustration with chatbot failures, yet their responses differed. Those who customized their chatbots predominantly displayed retrying intentions, in contrast to those using pre-personalized chatbots, who primarily showed giving-up intentions. Moreover, we noted different customizations were preferred for task-oriented and social-oriented chatbots. This study suggests that user-participated customization has the potential to foster the Ikea effect. This effect is expected to mitigate negative experiences with chatbots.
Supplemental Material
MP4 File - Video Preview
Video Preview
- Download
- 8.68 MB
- Download
- 45.92 MB
- Transcript
References
[1]
Dennis Benner, Edona Elshan, Sofia Schöbel, and Andreas Janson. 2021. What do you mean? A Review on Recovery Strategies to Overcome Conversational Breakdowns of Conversational Agents. In Proceedings of the 42nd International Conference on Information Systems, ICIS 2021, Building Sustainability and Resilience with IS: A Call for Action, Austin, TX, USA, December 12-15, 2021, Joe S. Valacich, Anitesh Barua, Ryan T. Wright, Atreyi Kankanhalli, Xitong Li, and Shaila Miranda (Eds.). Association for Information Systems. https://aisel.aisnet.org/icis2021/hci_robot/hci_robot/13
[2]
Lorentsa Gkinko and Amany R. Elbanna. 2022. Hope, tolerance and empathy: employees’ emotions when using an AI-enabled chatbot in a digitalised workplace. Inf. Technol. People 35, 6 (2022), 1714–1743. https://doi.org/10.1108/ITP-04-2021-0328
[3]
Sherrie YX Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly (2006), 941–960.
[4]
Cong Li. 2016. When does web-based personalization really work? The distinction between actual personalization and perceived personalization. Comput. Hum. Behav. 54 (2016), 25–33. https://doi.org/10.1016/J.CHB.2015.07.049
[5]
Mingming Liu, Qicheng Ding, Yu Zhang, Guoguang Zhao, Changjian Hu, Jiangtao Gong, Penghui Xu, Yu Zhang, Liuxin Zhang, and Qianying Wang. 2020. Cold Comfort Matters-How Channel-Wise Emotional Strategies Help in a Customer Service Chatbot. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–7.
[6]
Xiaojuan Ma, Emily P. Yang, and Pascale Fung. 2019. Exploring Perceived Emotional Intelligence of Personality-Driven Virtual Agents in Handling User Challenges. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, Ling Liu, Ryen W. White, Amin Mantrach, Fabrizio Silvestri, Julian J. McAuley, Ricardo Baeza-Yates, and Leila Zia (Eds.). ACM, 1222–1233. https://doi.org/10.1145/3308558.3313400
[7]
Yukun Ma, Khanh Linh Nguyen, Frank Z. Xing, and Erik Cambria. 2020. A survey on empathetic dialogue systems. Inf. Fusion 64 (2020), 50–70. https://doi.org/10.1016/J.INFFUS.2020.06.011
[8]
Amama Mahmood, Jeanie W. Fung, Isabel Won, and Chien-Ming Huang. 2022. Owning Mistakes Sincerely: Strategies for Mitigating AI Errors. In CHI ’22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022 - 5 May 2022, Simone D. J. Barbosa, Cliff Lampe, Caroline Appert, David A. Shamma, Steven Mark Drucker, Julie R. Williamson, and Koji Yatani (Eds.). ACM, 578:1–578:11. https://doi.org/10.1145/3491102.3517565
[9]
Michael I Norton, Daniel Mochon, and Dan Ariely. 2012. The IKEA effect: When labor leads to love. Journal of consumer psychology 22, 3 (2012), 453–460.
[10]
Giulia Pavone, Lars Meyer-Waarden, and Andreas Munzel. 2023. Rage against the machine: experimental insights into customers’ negative emotional responses, attributions of responsibility, and coping strategies in artificial intelligence–based service failures. Journal of Interactive Marketing 58, 1 (2023), 52–71.
[11]
Michael Shumanov and Lester Johnson. 2021. Making conversations with chatbots more personalized. Computers in Human Behavior 117 (2021), 106627.
[12]
Amy K Smith, Ruth N Bolton, and Janet Wagner. 1999. A model of customer satisfaction with service encounters involving failure and recovery. Journal of marketing research 36, 3 (1999), 356–372.
[13]
S Shyam Sundar and Sampada S Marathe. 2010. Personalization versus customization: The importance of agency, privacy, and power usage. Human communication research 36, 3 (2010), 298–322.
[14]
Rebecca Wald, Evelien Heijselaar, and Tibor Bosse. 2021. Make your own: The Potential of Chatbot Customization for the Development of User Trust. In Adjunct Publication of the 29th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2021, Utrecht, The Netherlands, June 21-25, 2021, Judith Masthoff, Eelco Herder, Nava Tintarev, and Marko Tkalcic (Eds.). ACM, 382–387. https://doi.org/10.1145/3450614.3463600
[15]
Rebecca Wald, Evelien Heijselaar, and Tibor Bosse. 2021. Make your own: The potential of chatbot customization for the development of user trust. In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. 382–387.
[16]
Siran Wang, Qiang Yan, and Lingli Wang. 2023. Task-oriented vs. social-oriented: Chatbot communication styles in electronic commerce service recovery. Electronic Commerce Research (2023), 1–33.
[17]
Nima Zargham, Dmitry Alexandrovsky, Jan Erich, Nina Wenig, and Rainer Malaka. 2022. "I Want It That Way": Exploring Users’ Customization and Personalization Preferences for Home Assistants. In CHI ’22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022 - 5 May 2022, Extended Abstracts, Simone D. J. Barbosa, Cliff Lampe, Caroline Appert, and David A. Shamma (Eds.). ACM, 270:1–270:8. https://doi.org/10.1145/3491101.3519843
Index Terms
- Exploring the Impact of User-Participated Customization in Experiencing Chatbot Failure
Recommendations
The role of choice and customization on users' interaction with embodied conversational agents: effects on perception and performance
CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe performed an empirical study exploring people's interactions with an embodied conversational agent (ECA) while performing two tasks. Conditions varied with respect to 1) whether participants were allowed to choose an agent and its characteristics and ...
QoS customization in distributed object systems
Special issue: MiddlewareApplications built on networked collections of computers are increasingly using distributed object platforms such as CORBA, Java Remote Method Invocation (RMI), and DCOM to standardize object interactions. With this increased use comes the increased ...
The role of user differences in customization: a case study in personalization for infovis-based content
IUI '19: Proceedings of the 24th International Conference on Intelligent User InterfacesAlthough there is extensive evidence that personalization of interactive systems can improve the user's experience and satisfaction, it is also known that the two main approaches to deliver personalization, namely via customization or system-driven ...
Comments
Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Work in progress
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI 2025
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 597Total Downloads
- Downloads (Last 12 months)597
- Downloads (Last 6 weeks)87
Reflects downloads up to 26 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format