research-article

Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems

Authors:

Johannes Kunkel,

Catalin-Mihai Barbu,

Jürgen ZieglerAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 487, Pages 1 - 12

https://doi.org/10.1145/3290605.3300717

Published: 02 May 2019 Publication History

Abstract

Trust in a Recommender System (RS) is crucial for its overall success. However, it remains underexplored whether users trust personal recommendation sources (i.e. other humans) more than impersonal sources (i.e. conventional RS), and, if they do, whether the perceived quality of explanation provided account for the difference. We conducted an empirical study in which we compared these two sources of recommendations and explanations. Human advisors were asked to explain movies they recommended in short texts while the RS created explanations based on item similarity. Our experiment comprised two rounds of recommending. Over both rounds the quality of explanations provided by users was assessed higher than the quality of the system's explanations. Moreover, explanation quality significantly influenced perceived recommendation quality as well as trust in the recommendation source. Consequently, we suggest that RS should provide richer explanations in order to increase their perceived recommendation quality and trustworthiness.

References

[1]

James Bennett and Stan Lanning. 2007. The netflix prize. In Proceedings of KDD Cup and Workshop (KDDCup '07), Vol. 2007. San Jose, CA, 3--6.

[2]

Shlomo Berkovsky, Ronnie Taib, and Dan Conway. 2017. How to Recommend?: User Trust Factors in Movie Recommender Systems. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (IUI '17). ACM, New York, NY, 287--300.

Digital Library

[3]

M. Bilgic and Raymond J. Mooney. 2005. Explaining recommendations: satisfaction vs. promotion. In Proceedings of Beyond Personalization Workshop, IUI.

[4]

André Calero Valdez, Martina Ziefle, and Katrien Verbert. 2016. HCI for Recommender Systems: The Past, the Present and the Future. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys '16). ACM, New York, NY, 123--126.

Digital Library

[5]

Shuo Chang, F. Maxwell Harper, and Loren Gilbert Terveen. 2016. Crowd-Based Personalized Natural Language Explanations for Recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys '16). ACM, New York, NY, 175--182.

Digital Library

[6]

Jaewon Choi, Hong Joo Lee, and Yong Cheol Kim. 2009. The influence of social presence on evaluating personalized recommender systems. In Pacific Asia Conference on Information Systems (PACIS).

[7]

Jacob Cohen. 1988. Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum, Hillsdale, NJ.

[8]

Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and UserAdapted Interaction 18, 5 (2008), 455--496.

Digital Library

[9]

Tim Donkers, Benedikt Loepp, and Jürgen Ziegler. 2018. Explaining Recommendations by Means of User Reviews. In Proceedings of the 1st Workshop on Explainable Smart Systems (ExSS '18). http://ceur-ws. org/Vol-2068/exss8.pdf

[10]

A. Felfernig and B. Gula. 2006. An Empirical Study on Consumer Behavior in the Interaction with Knowledge-based Recommender Applications. In The 8th IEEE International Conference on E-Commerce Technology and The 3rd IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services (CEC/EEE '06). 159--169.

Digital Library

[11]

David A Freedman. 2006. On The So-Called "Huber Sandwich Estimator" and "Robust Standard Errors". The American Statistician 60, 4 (2006), 299--302.

[12]

Fatih Gedikli, Dietmar Jannach, and Mouzhi Ge. 2014. How should I explain? A comparison of different explanation types for recommender systems. International Journal of Human-Computer Studies 72, 4 (2014), 367--382.

Digital Library

[13]

David Gefen. 1997. Building Users' Trust in Freeware Providers and the Effects of This Trust on Users' Perceptions of Usefulness, Ease of Use and Intended Use of Freeware. Ph.D. Dissertation. Atlanta, GA.

Digital Library

[14]

Carlos A. Gomez-Uribe and Neil Hunt. 2015. The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Transactions on Management Information Systems 6, 4 (2015), 13:1--13:19.

Digital Library

[15]

Asela Gunawardana and Guy Shani. 2015. Evaluating Recommender Systems. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 265--308.

[16]

Ido Guy. 2015. Social Recommender Systems. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 511--543.

[17]

Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining Collaborative Filtering Recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW '00). ACM, New York, NY, 241--250.

Digital Library

[18]

Miguel Angel Hernán. 2004. A definition of causal effect for epidemiological research. Journal of Epidemiology & Community Health 58, 4 (2004), 265--271.

[19]

Anthony Jameson, Martijn C. Willemsen, Alexander Felfernig, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro, and Li Chen. 2015. Human Decision Making and Recommender Systems. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 611--648.

[20]

Michael Jugovac and Dietmar Jannach. 2017. Interacting with Recommenders--Overview and Research Directions. ACM Transactions on Interactive Intelligent Systems (TiiS) 7, 3 (2017), 10:1--10:46.

Digital Library

[21]

Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the User Experience of Recommender Systems. User Modeling and User-Adapted Interaction 22, 4--5 (Oct. 2012), 441--504.

Digital Library

[22]

Sherrie Y. X. Komiak and Izak Benbasat. 2006. The Effects of Personalizaion and Familiarity on Trust and Adoption of Recommendation Agents. MIS Quarterly 30, 4 (2006), 941--960.

Digital Library

[23]

Joseph A. Konstan and John Riedl. 2012. Recommender systems: from algorithms to user experience. User Modeling and UserAdapted Interaction 22, 1--2 (2012), 101--123.

Digital Library

[24]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer 42, 8 (2009), 30--37.

Digital Library

[25]

Vinod Krishnan, Pradeep Kumar Narayanashetty, Mukesh Nathan, Richard T. Davies, and Joseph A. Konstan. 2008. Who Predicts Better?: Results from an Online Study Comparing Humans and an Online Recommender System. In Proceedings of the 2008 ACM Conference on Recommender Systems (RecSys '08). ACM, New York, NY, 211--218.

Digital Library

[26]

Johannes Kunkel, Tim Donkers, Catalin-Mihai Barbu, and Jürgen Ziegler. 2018. Trust-Related Effects of Expertise and Similarity Cues in Human-Generated Recommendations. In Companion Proceedings of the 23rd International on Intelligent User Interfaces: 2nd Workshop on Theory-Informed User Modeling for Tailoring and Personalizing Interfaces (HUMANIZE). http://ceur-ws.org/Vol-2068/humanize5.pdf

[27]

Johannes Kunkel, Benedikt Loepp, and Jürgen Ziegler. 2017. A 3D Item Space Visualization for Presenting and Manipulating User Preferences in Collaborative Filtering. In Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion (IUI '17). ACM, New York, NY, 3--15.

Digital Library

[28]

John D. Lee and Katrina A. See. 2004. Trust in Automation: Designing for Appropriate Reliance. Human Factors 46, 1 (2004), 50--80.

[29]

Benedikt Loepp, Tim Donkers, Timm Kleemann, and Jürgen Ziegler. 2018. Impact of Item Consumption on Assessment of Recommendations in User Studies. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, New York, NY, 49-53.

Digital Library

[30]

D. Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and Validating Trust Measures for e-Commerce: An Integrative Typology. Information Systems Research 13, 3 (2002), 334--359.

Digital Library

[31]

D. Harrison McKnight, Larry L. Cummings, and Norman L. Chervany. 1998. Initial Trust Formation in New Organizational Relationships. Academy of Management Review 23, 3 (1998), 473--490.

[32]

Sean M. McNee, John Riedl, and Joseph A. Konstan. 2006. Being Accurate is Not Enough: How Accuracy Metrics Have Hurt Recommender Systems. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (CHI EA '06). ACM, New York, NY, 1097--1101.

Digital Library

[33]

Bengt Muthén. 1984. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika 49, 1 (1984), 115--132.

[34]

Umberto Panniello, Michele Gorgoglione, and Alexander Tuzhilin. 2015. Research Note-In CARSs We Trust: How Context-Aware Recommendations Affect Customers' Trust and Other Business Performance Measures of Recommender Systems. Information Systems Research 27, 1 (2015), 182--196.

[35]

Judea Pearl. 2012. The Mediation Formula: A Guide to the Assessment of Causal Pathways in Nonlinear Models. Causality: Statistical Perspectives and Applications (2012), 151--179.

[36]

Pearl Pu and Li Chen. 2007. Trust-inspiring explanation interfaces for recommender systems. Knowledge-Based Systems 20, 6 (2007), 542--556.

Digital Library

[37]

Pearl Pu, Li Chen, and Rong Hu. 2012. Evaluating recommender systems from the user's perspective: survey of the state of the art. User Modeling and User-Adapted Interaction 22, 4--5 (2012), 317--355.

Digital Library

[38]

Francesco Ricci, Lior Rokach, and Bracha Shapira. 2015. Recommender Systems: Introduction and Challenges. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 1--34.

[39]

Yves Rosseel. 2012. lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software 48, 2 (2012), 1--36. http: //www.jstatsoft.org/v48/i02/

[40]

Donald B Rubin. 1990. Formal mode of statistical inference for causal effects. Journal of statistical planning and inference 25, 3 (1990), 279--292.

[41]

Sylvain Senecal and Jacques Nantel. 2004. The influence of online product recommendations on consumers' online choices. Journal of Retailing 80, 2 (2004), 159--169.

[42]

Rashmi Sinha and Kirsten Swearingen. 2002. The Role of Transparency in Recommender Systems. In Extended Abstracts on Human Factors in Computing Systems (CHI EA '02). ACM, New York, NY, 830--831.

Digital Library

[43]

B. Smith and G. Linden. 2017. Two Decades of Recommender Systems at Amazon.com. Internet Computing, IEEE 21, 3 (2017), 12--18.

Digital Library

[44]

Donnavieve Smith, Satya Menon, and K. Sivakumar. 2005. Online peer and editorial recommendations, trust, and choice in virtual markets. Journal of Interactive Marketing 19, 3 (2005), 15--37.

[45]

Nava Tintarev and Judith Masthoff. 2012. Evaluating the effectiveness of explanations for recommender systems. User Modeling and UserAdapted Interaction 22, 4 (2012), 399--439.

Digital Library

[46]

Nava Tintarev and Judith Masthoff. 2015. Explaining Recommendations: Design and Evaluation. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 353--382.

[47]

Patricia Victor, Martine de Cock, and Chris Cornelis. 2011. Trust and Recommendations. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor (Eds.). Springer US, Boston, MA, 645--675.

[48]

J Christopher Westland. 2010. Lower bounds on sample size in structural equation modeling. Electronic Commerce Research and Applications 9, 6 (2010), 476--487.

Digital Library

[49]

Sherrie Xiao and Izak Benbasat. 2003. The Formation of Trust and Distrust in Recommendation Agents in Repeated Interactions: A Processtracing Analysis. In Proceedings of the 5th International Conference on Electronic Commerce (ICEC '03). ACM, New York, NY, 287--293.

Digital Library

[50]

Jingjing Zhang and Shawn P. Curley. 2018. Exploring Explanation Effects on Consumers' Trust in Online Recommender Agents. International Journal of Human--Computer Interaction 34, 5 (2018), 421--432.

Cited By

Cheng XPeng G(2024)Study on the Behavioral Motives of Algorithmic Avoidance in Intelligent Recommendation SystemsJournal of Global Information Management10.4018/JGIM.35285732:1(1-22)Online publication date: 17-Sep-2024
https://dl.acm.org/doi/10.4018/JGIM.352857
Naveed SStevens GRobin-Kern D(2024)An Overview of the Empirical Evaluation of Explainable AI (XAI): A Comprehensive Guideline for User-Centered Evaluation in XAIApplied Sciences10.3390/app14231128814:23(11288)Online publication date: 3-Dec-2024
https://doi.org/10.3390/app142311288
Zhu YLong YWang HLee KZhang LWang S(2024)Digital Behavior Change Intervention Designs for Habit Formation: Systematic ReviewJournal of Medical Internet Research10.2196/5437526(e54375)Online publication date: 24-May-2024
https://doi.org/10.2196/54375
Show More Cited By

Index Terms

Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

On the Negative Perception of Cross-domain Recommendations and Explanations
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recommender systems typically operate within a single domain, for example, recommending books based on users' reading habits. If such data is unavailable, it may be possible to make cross-domain recommendations and recommend books based on user ...
Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

User-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
How should I explain? A comparison of different explanation types for recommender systems

Recommender systems help users locate possible items of interest more quickly by filtering and ranking them in a personalized way. Some of these systems provide the end user not only with such a personalized item list but also with an explanation which ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

108
Total Citations
View Citations
2,127
Total Downloads

Downloads (Last 12 months)335
Downloads (Last 6 weeks)40

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cheng XPeng G(2024)Study on the Behavioral Motives of Algorithmic Avoidance in Intelligent Recommendation SystemsJournal of Global Information Management10.4018/JGIM.35285732:1(1-22)Online publication date: 17-Sep-2024
https://dl.acm.org/doi/10.4018/JGIM.352857
Naveed SStevens GRobin-Kern D(2024)An Overview of the Empirical Evaluation of Explainable AI (XAI): A Comprehensive Guideline for User-Centered Evaluation in XAIApplied Sciences10.3390/app14231128814:23(11288)Online publication date: 3-Dec-2024
https://doi.org/10.3390/app142311288
Zhu YLong YWang HLee KZhang LWang S(2024)Digital Behavior Change Intervention Designs for Habit Formation: Systematic ReviewJournal of Medical Internet Research10.2196/5437526(e54375)Online publication date: 24-May-2024
https://doi.org/10.2196/54375
Ling SZhang YDu N(2024)More Is Not Always Better: Impacts of AI-Generated Confidence and Explanations in Human–Automation InteractionHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/00187208241234810Online publication date: 4-Mar-2024
https://doi.org/10.1177/00187208241234810
Lyu QApidianaki MCallison-Burch C(2024)Towards Faithful Model Explanation in NLP: A SurveyComputational Linguistics10.1162/coli_a_0051150:2(657-723)Online publication date: 1-Jun-2024
https://doi.org/10.1162/coli_a_00511
Ferguson SAoyagui PRizvi RKim YKuzminykh A(2024)The Explanation That Hits Home: The Characteristics of Verbal Explanations That Affect Human Perception in Subjective Decision-MakingProceedings of the ACM on Human-Computer Interaction10.1145/36870568:CSCW2(1-37)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687056
Dix ALuyten KMayer SPalanque PPanizzi ESpano LZiegler J(2024)Second Workshop on Engineering Interactive Systems Embedding AI TechnologiesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3662837(103-107)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3662837
Ge YLiu SFu ZTan JLi ZXu SLi YXian YZhang Y(2024)A Survey on Trustworthy Recommender SystemsACM Transactions on Recommender Systems10.1145/36528913:2(1-68)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3652891
Lim HCho JKim TPark JShin HChoi SPark SLee KKim JLee MHong H(2024)Co-Creating Question-and-Answer Style Articles with Large Language Models for Research PromotionProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660705(975-994)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660705
Muijlwijk HWillemsen MSmyth BIJsselsteijn W(2024)Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon RunningProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645205(245-258)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645205
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents