Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3301275.3302313acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
research-article

To explain or not to explain: the effects of personal characteristics when explaining music recommendations

Published: 17 March 2019 Publication History

Abstract

Recommender systems have been increasingly used in online services that we consume daily, such as Facebook, Netflix, YouTube, and Spotify. However, these systems are often presented to users as a "black box", i.e. the rationale for providing individual recommendations remains unexplained to users. In recent years, various attempts have been made to address this black box issue by providing textual explanations or interactive visualisations that enable users to explore the provenance of recommendations. Among other things, results demonstrated benefits in terms of precision and user satisfaction. Previous research had also indicated that personal characteristics such as domain knowledge, trust propensity and persistence may also play an important role on such perceived benefits. Yet, to date, little is known about the effects of personal characteristics on explaining recommendations. To address this gap, we developed a music recommender system with explanations and conducted an online study using a within-subject design. We captured various personal characteristics of participants and administered both qualitative and quantitative evaluation methods. Results indicate that personal characteristics have significant influence on the interaction and perception of recommender systems, and that this influence changes by adding explanations. For people with a low need for cognition are the explained recommendations the most beneficial. For people with a high need for cognition, we observed that explanations could create a lack of confidence. Based on these results, we present some design implications for explaining recommendations.

Supplementary Material

MP4 File (p397-millecamp.mp4)

References

[1]
Azzah Al-Maskari and Mark Sanderson. 2011. The effect of user characteristics on search effectiveness in information retrieval. Information Processing & Management 47, 5 (2011), 719--729.
[2]
Ivana Andjelkovic, Denis Parra, and John O'Donovan. 2016. Moodplay: Interactive Mood-based Music Discovery and Recommendation. In Proc. of UMAP '16. ACM, 275--279.
[3]
Nuray M Aykin and Turgut Aykin. 1991. Individual differences in human-computer interaction. Computers & industrial engineering 20, 3 (1991), 373--379.
[4]
Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67, 1 (2015), 1--48.
[5]
Svetlin Bostandjiev, John O'Donovan, and Tobias Höllerer. 2012. TasteWeights: a visual interactive hybrid recommender system. In Proc. of RecSys'12. ACM, 35--42.
[6]
Jeremy Boy, Ronald A Rensink, Enrico Bertini, and Jean-Daniel Fekete. 2014. A principled way of assessing visualization literacy. IEEE transactions on visualization and computer graphics 20, 12 (2014), 1963--1972.
[7]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.
[8]
Peter Brusilovsky and Eva Millán. 2007. User models for adaptive hypermedia and adaptive educational systems. In The adaptive web. Springer, 3--53.
[9]
Andrea Bunt, Joanna McGrenere, and Cristina Conati. 2007. Understanding the utility of rationale in a mixed-initiative system for GUI customization. In International Conference on User Modeling. Springer, 147--156.
[10]
John T Cacioppo, Richard E Petty, and Chuan Feng Kao. 1984. The efficient assessment of need for cognition. Journal of personality assessment 48, 3 (1984), 306--307.
[11]
Giuseppe Carenini, Cristina Conati, Enamul Hoque, Ben Steichen, Dereck Toker, and James Enns. 2014. Highlighting interventions and user differences: informing adaptive information visualization support. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 1835--1844.
[12]
Li Chen and Pearl Pu. 2005. Trust building in recommender agents. In Proceedings of the Workshop on Web Personalization, Recommender Systems and Intelligent User Interfaces at the 2nd International Conference on E-Business and Telecommunication Networks. Citeseer, 135--145.
[13]
Mei C Chuah. 1998. Dynamic aggregation with circular visual designs. In Information Visualization, 1998. Proceedings. IEEE Symposium on. IEEE, 35--43.
[14]
Cristina Conati, Giuseppe Carenini, Enamul Hoque, Ben Steichen, and Dereck Toker. 2014. Evaluating the impact of user characteristics and different layouts on an interactive visualization for decision making. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 371--380.
[15]
Cristina Conati, Giuseppe Carenini, Dereck Toker, and Sébastien Lallé. 2015. Towards user-adaptive information visualization. In Proc. of AAAI '15. AAAI Press, 4100--4106.
[16]
Ipshita Dewan and Pierre Benckendorff. 2013. Impact of Tech Savviness and impulsiveness on the mobile information search behaviour of young travellers. Information and communications technologies in tourism (2013).
[17]
Gitta O Domik and Bernd Gutkauf. 1994. User modeling for adaptive visualization systems. In Visualization, 1994., Visualization'94, Proceedings., IEEE Conference on. IEEE, 217--223.
[18]
G. Fournier. 2018 (Retrieved on September 16, 2018). Locus of Control. Psych Central. https://psychcentral.com/encyclopedia/locus-of-control/
[19]
Roger W Geyer. 2009. Developing the internet-savviness (IS) scale: Investigating the relationships between internet use and academically talented middle school youth. RMLE Online 32, 5 (2009), 1--20.
[20]
Liang Gou, Fang You, Jun Guo, Luqi Wu, and Xiaolong Luke Zhang. 2011. Sfviz: interest-based friends exploration and recommendation in social networks. In Proc. VINCI '11. ACM, 15.
[21]
David M Greenberg, Daniel Müllensiefen, Michael E Lamb, and Peter J Rentfrow. 2015. Personality predicts musical sophistication. Journal of Research in Personality 58 (2015), 154--158.
[22]
Brynjar Gretarsson, John O'Donovan, Svetlin Bostandjiev, Christopher Hall, and Tobias Höllerer. 2010. Smallworlds: visualizing social recommendations. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 833--842.
[23]
Jonathan L Herlocker, Joseph A Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. ACM, 241--250.
[24]
Yosef Hochberg and Yoav Benjamini. 1990. More powerful procedures for multiple significance testing. Statistics in medicine 9, 7 (1990), 811--818.
[25]
Yucheng Jin, Karsten Seipp, Erik Duval, and Katrien Verbert. 2016. Go with the flow: effects of transparency and user control on targeted advertising using flow charts. In Proc. of AVI '16. ACM, 68--75.
[26]
Yucheng Jin, Nava Tintarev, and Katrien Verbert. 2018. Effects of individual traits on diversity-aware music recommender user interfaces. In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization. ACM, 291--299.
[27]
Yucheng Jin, Nava Tintarev, and Katrien Verbert. 2018. Effects of personal characteristics on music recommender systems with different levels of controllability. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 13--21.
[28]
Roy PC Kessels, Martine JE Van Zandvoort, Albert Postma, L Jaap Kappelle, and Edward HF De Haan. 2000. The Corsi block-tapping task: standardization and normative data. Applied neuropsychology 7, 4 (2000), 252--258.
[29]
Bart P Knijnenburg, Niels JM Reijmer, and Martijn C Willemsen. 2011. Each to his own: how different users call for different interaction methods in recommender systems. In Proc. of RecSys'11. ACM, 141--148.
[30]
Bart P Knijnenburg, Martijn C Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction 22, 4-5 (2012), 441--504.
[31]
Alexandra Kuznetsova, Per B. Brockhoff, and Rune H. B. Christensen. 2017. lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software 82, 13 (2017), 1--26.
[32]
Sébastien Lallé, Cristina Conati, and Giuseppe Carenini. 2017. Impact of Individual Differences on User Experience with a Visualization Interface for Public Engagement. In Proc. of UMAP '17. ACM, 247--252.
[33]
Moira Maguire and Brid Delahunt. 2017. Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars. AISHE-J: The All Ireland Journal of Teaching and Learning in Higher Education 9, 3 (2017).
[34]
Martijn Millecamp, Nyi Nyi Htun, YuchengJin, and Katrien Verbert. 2018. Controlling Spotify recommendations: effects of personal characteristics on music recommender user Interfaces. In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization. ACM, 101--109.
[35]
Akira Miyake and Priti Shah. 1999. Models of working memory: Mechanisms of active maintenance and executive control. Cambridge University Press.
[36]
Daniel Müllensiefen, Bruno Gingras, Jason Musil, and Lauren Stewart. 2014. The musicality of non-musicians: an index for assessing musical sophistication in the general population. PloS one 9, 2 (2014), e89642.
[37]
John O'Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1085--1088.
[38]
Thomas V Perneger. 1998. What's wrong with Bonferroni adjustments. Bmj 316, 7139 (1998), 1236--1238.
[39]
Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems. ACM, 157--164.
[40]
R Core Team. 2017. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/
[41]
Julian B Rotter. 1966. Generalized expectancies for internal versus external control of reinforcement. Psychological monographs: General and applied 80, 1 (1966), 1.
[42]
Rashmi Sinha and Kirsten Swearingen. 2002. The role of transparency in recommender systems. In CHI'02 extended abstracts on Human factors in computing systems. ACM, 830--831.
[43]
Nava Tintarev. 2017. Presenting Diversity Aware Recommendations: Making Challenging News Acceptable. In Proc. of FATREC 17'.
[44]
Nava Tintarev and Judith Masthoff. 2007. A survey of explanations in recommender systems. In Data Engineering Workshop, 2007 IEEE 23rd International Conference on. IEEE, 801--810.
[45]
Nava Tintarev and Judith Masthoff. 2016. Effects of Individual Differences in Working Memory on Plan Presentational Choices. Frontiers in psychology 7 (2016).
[46]
Dereck Toker, Cristina Conati, Giuseppe Carenini, and Mona Haraty. 2012. Towards adaptive information visualization: on the influence of user characteristics. In International Conference on User Modeling, Adaptation, and Personalization. Springer, 274--285.
[47]
Stephanie Tom Tong, Elena F Corriero, Robert G Matheny, and Jeffrey T Hancock. 2018. Online Daters' Willingness to Use Recommender Technology for Mate Selection Decisions. In Proceedings of the 5th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems co-located with ACM Conference on Recommender Systems (RecSys 2018). ACM, 45--52.
[48]
Chun-Hua Tsai and Peter Brusilovsky. 2017. Enhancing Recommendation Diversity Through a Dual Recommendation Interface. In Proc. of RecSys IntRS'17. 10.
[49]
Chun-Hua Tsai and Peter Brusilovsky. 2018. Beyond the Ranked List: User-Driven Exploration and Diversification of Social Recommendation. In 23rd International Conference on Intelligent User Interfaces. ACM, 239--250.
[50]
AJAM Van Deursen, Ellen J Helsper, and R Eynon. 2014. Measuring digital skills. From digital skills to tangible outcomes. Project Report. Recuperado de: www.oii.ox.ac.uk/research/projects (2014).
[51]
Katrien Verbert, Denis Parra, Peter Brusilovsky, and Erik Duval. 2013. Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the 2013 international conference on Intelligent user interfaces. ACM, 351--362.
[52]
Bodo Winter. 2013. A very basic tutorial for performing linear mixed effects analyses. arXiv preprint arXiv:1308.5499 (2013).

Cited By

View all
  • (2024)An Overview of the Empirical Evaluation of Explainable AI (XAI): A Comprehensive Guideline for User-Centered Evaluation in XAIApplied Sciences10.3390/app14231128814:23(11288)Online publication date: 3-Dec-2024
  • (2024)Citizens’ trust in AI-enabled government systemsInformation Polity10.3233/IP-23006529:3(293-312)Online publication date: 27-Aug-2024
  • (2024)Surveying More Than Two Decades of Music Information Retrieval Research on PlaylistsACM Transactions on Intelligent Systems and Technology10.1145/368839815:6(1-68)Online publication date: 12-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces
March 2019
713 pages
ISBN:9781450362726
DOI:10.1145/3301275
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. explanations
  2. music
  3. need for cognition
  4. personal characteristics
  5. recommender system
  6. spotify
  7. user characteristics

Qualifiers

  • Research-article

Conference

IUI '19
Sponsor:

Acceptance Rates

IUI '19 Paper Acceptance Rate 71 of 282 submissions, 25%;
Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)307
  • Downloads (Last 6 weeks)30
Reflects downloads up to 26 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)An Overview of the Empirical Evaluation of Explainable AI (XAI): A Comprehensive Guideline for User-Centered Evaluation in XAIApplied Sciences10.3390/app14231128814:23(11288)Online publication date: 3-Dec-2024
  • (2024)Citizens’ trust in AI-enabled government systemsInformation Polity10.3233/IP-23006529:3(293-312)Online publication date: 27-Aug-2024
  • (2024)Surveying More Than Two Decades of Music Information Retrieval Research on PlaylistsACM Transactions on Intelligent Systems and Technology10.1145/368839815:6(1-68)Online publication date: 12-Aug-2024
  • (2024)When to Explain? Exploring the Effects of Explanation Timing on User Perceptions and Trust in AI systemsProceedings of the Second International Symposium on Trustworthy Autonomous Systems10.1145/3686038.3686066(1-17)Online publication date: 16-Sep-2024
  • (2024)Slide to Explore 'What If': An Analysis of Explainable InterfacesAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685416(1-6)Online publication date: 13-Oct-2024
  • (2024)Visualization for Recommendation Explainability: A Survey and New PerspectivesACM Transactions on Interactive Intelligent Systems10.1145/367227614:3(1-40)Online publication date: 11-Jun-2024
  • (2024)A Survey of Music Recommendation SystemsProceedings of the 5th International Conference on Computer Information and Big Data Applications10.1145/3671151.3671243(507-519)Online publication date: 26-Apr-2024
  • (2024)What Did I Say Again? Relating User Needs to Search Outcomes in Conversational CommerceProceedings of Mensch und Computer 202410.1145/3670653.3670680(129-139)Online publication date: 1-Sep-2024
  • (2024)Fairness and Transparency in Music Recommender Systems: Improvements for ArtistsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688024(1368-1375)Online publication date: 8-Oct-2024
  • (2024)Promoting Green Fashion Consumption in Recommender SystemsAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664922(50-54)Online publication date: 27-Jun-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media