research-article

Critiquing for Music Exploration in Conversational Recommender Systems

Authors:

Li ChenAuthors Info & Claims

IUI '21: Proceedings of the 26th International Conference on Intelligent User Interfaces

Pages 480 - 490

https://doi.org/10.1145/3397481.3450657

Published: 14 April 2021 Publication History

Abstract

Dialogue-based conversational recommender systems allow users to give language-based feedback on the recommended item, which has great potential for supporting users to explore the space of recommendations through conversation. In this work, we consider incorporating critiquing techniques into conversational systems to facilitate users’ exploration of music recommendations. Thus, we have developed a music chatbot with three system variants, which are respectively featured with three different critiquing techniques, i.e., user-initiated critiquing (UC), progressive system-suggested critiquing (Progressive SC), and cascading system-suggested critiquing (Cascading SC). We conducted a between-subject study (N=107) to compare these three types of systems with regards to music exploration in terms of user perception and user interaction. Results show that both UC and SC are useful for music exploration, while users perceive higher diversity of recommendations with the system that offers Cascading SC and perceive more serendipitous with the system that offers Progressive SC. In addition, we find that the critiquing techniques significantly moderate the relationships between some interaction metrics (e.g., number of listened songs, number of dialogue turns) and users’ perceived helpfulness and serendipity during music exploration.

References

[1]

Ivana Andjelkovic, Denis Parra, and John O’Donovan. 2016. Moodplay: Interactive Mood-based Music Discovery and Recommendation. In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization (Halifax, Nova Scotia, Canada) (UMAP ’16). ACM, New York, NY, USA, 275–279. https://doi.org/10.1145/2930238.2930280

Digital Library

[2]

Ivana Andjelkovic, Denis Parra, and John O’Donovan. 2019. Moodplay: Interactive music recommendation based on Artists’ mood similarity. International Journal of Human-Computer Studies 121 (2019), 142–159.

[3]

Wanling Cai and Li Chen. 2020. Predicting User Intents and Satisfaction with Dialogue-Based Conversational Recommendations. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (Genoa, Italy) (UMAP ’20). Association for Computing Machinery, New York, NY, USA, 33–42. https://doi.org/10.1145/3340631.3394856

Digital Library

[4]

Li Chen and Pearl Pu. 2006. Evaluating Critiquing-based Recommender Agents. In Proceedings of the 21st National Conference on Artificial Intelligence - Volume 1 (Boston, Massachusetts) (AAAI’06). AAAI Press, 157–162. http://dl.acm.org/citation.cfm?id=1597538.1597564

Digital Library

[5]

Li Chen and Pearl Pu. 2007. Preference-based organization interfaces: aiding user critiques in recommender systems. In International Conference on User Modeling. Springer, 77–86.

Digital Library

[6]

Li Chen and Pearl Pu. 2010. Eye-Tracking Study of User Behavior in Recommender Interfaces. In Proceedings of the 18th International Conference on User Modeling, Adaptation, and Personalization(Big Island, HI) (UMAP’10). Springer-Verlag, Berlin, Heidelberg, 375–380. https://doi.org/10.1007/978-3-642-13470-8_35

Digital Library

[7]

Li Chen and Pearl Pu. 2012. Critiquing-based Recommenders: Survey and Emerging Trends. User Modeling and User-Adapted Interaction 22, 1-2 (April 2012), 125–150. https://doi.org/10.1007/s11257-011-9108-6

Digital Library

[8]

Franz Faul, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior research methods 41, 4 (2009), 1149–1160.

[9]

Neil Hurley and Mi Zhang. 2011. Novelty and Diversity in Top-N Recommendation – Analysis and Evaluation. ACM Trans. Internet Technol. 10, 4, Article 14 (March 2011), 30 pages. https://doi.org/10.1145/1944339.1944341

Digital Library

[10]

Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 895–906. https://doi.org/10.1145/3196709.3196735

Digital Library

[11]

Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2020. A Survey on Conversational Recommender Systems. arXiv preprint arXiv:2004.00646(2020).

[12]

Yucheng Jin, Wanling Cai, Li Chen, Nyi Nyi Htun, and Katrien Verbert. 2019. MusicBot: Evaluating Critiquing-Based Music Recommenders with Conversational Interaction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, 951–960. https://doi.org/10.1145/3357384.3357923

Digital Library

[13]

Mohsen Kamalzadeh, Christoph Kralj, Torsten Möller, and Michael Sedlmair. 2016. TagFlip: active mobile music discovery with social tags. In Proceedings of the 21st International Conference on Intelligent User Interfaces. 19–30.

Digital Library

[14]

Marius Kaminskas and Derek Bridge. 2016. Diversity, serendipity, novelty, and coverage: a survey and empirical analysis of beyond-accuracy objectives in recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS) 7, 1(2016), 1–42.

Digital Library

[15]

Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding How People Use Natural Language to Ask for Recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (Como, Italy) (RecSys ’17). ACM, New York, NY, USA, 229–237. https://doi.org/10.1145/3109859.3109873

Digital Library

[16]

Jaya Kumar and Nava Tintarev. 2018. Using Visualizations to Encourage Blind-Spot Exploration. In Recsys workshop on Interfaces and Decision Making in Recommender Systems.

[17]

Wolfgang Lenhard and Alexandra Lenhard. 2014. Hypothesis Tests for Comparing Correlations. Psychometrica. https://doi.org/10.13140/RG.2.1.2954.1367

[18]

Mark Levy and Klaas Bosteels. 2010. Music recommendation and the long tail. In 1st Workshop On Music Recommendation And Discovery, ACM RecSys (Barcelona, Spain) (WOMRAD).

[19]

Chang Li, Haoyun Feng, and Maarten de Rijke. 2020. Cascading Hybrid Bandits: Online Learning to Rank for Relevance and Diversity. In Fourteenth ACM Conference on Recommender Systems (Virtual Event, Brazil) (RecSys ’20). Association for Computing Machinery, New York, NY, USA, 33–42. https://doi.org/10.1145/3383313.3412245

Digital Library

[20]

Yu Liang and Martijn C. Willemsen. 2019. Personalized Recommendations for Music Genre Exploration. In Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization (Larnaca, Cyprus) (UMAP ’19). ACM, New York, NY, USA, 276–284. https://doi.org/10.1145/3320435.3320455

Digital Library

[21]

Christian Matt, Alexander Benlian, Thomas Hess, and Christian Weiß. 2014. Escaping from the Filter Bubble? The Effects of Novelty and Serendipity on Users’ Evaluations of Online Recommendations. In Thirty Fifth International Conference on Information Systems.

[22]

Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti. 2017. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology 70 (2017), 153–163.

[23]

Zhenhui Peng, Yunhwan Kwon, Jiaan Lu, Ziming Wu, and Xiaojuan Ma. 2019. Design and Evaluation of Service Robot’s Proactivity in Decision-Making Support Process. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300328

Digital Library

[24]

James Reilly, Kevin McCarthy, Lorraine McGinty, 2004. Incremental critiquing. In Proc. of SGAI’04. Springer, 101–114.

[25]

Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. 2015. Recommender Systems Handbook(2nd ed.). Springer-Verlag.

Digital Library

[26]

Tobias Schnabel, Paul N. Bennett, Susan T. Dumais, and Thorsten Joachims. 2016. Using Shortlists to Support Decision Making and Improve Recommender System Performance. In Proceedings of the 25th International Conference on World Wide Web (Montréal, Québec, Canada) (WWW ’16). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 987–997. https://doi.org/10.1145/2872427.2883012

Digital Library

[27]

Tobias Schnabel, Paul N. Bennett, Susan T. Dumais, and Thorsten Joachims. 2018. Short-Term Satisfaction and Long-Term Coverage: Understanding How Users Tolerate Algorithmic Exploration. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (Marina Del Rey, CA, USA) (WSDM ’18). ACM, New York, NY, USA, 513–521. https://doi.org/10.1145/3159652.3159700

Digital Library

[28]

Hideo Shimazu. 2001. ExpertClerk: Navigating Shoppers’ Buying Process with the Combination of Asking and Proposing. In Proceedings of the 17th International Joint Conference on Artificial Intelligence - Volume 2 (Seattle, WA, USA) (IJCAI’01). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1443–1448. http://dl.acm.org/citation.cfm?id=1642194.1642287

[29]

Taavi T. Taijala, Martijn C. Willemsen, and Joseph A. Konstan. 2018. MovieExplorer: Building an Interactive Exploration Tool from Ratings and Latent Taste Spaces. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing (Pau, France) (SAC ’18). ACM, New York, NY, USA, 1383–1392. https://doi.org/10.1145/3167132.3167281

Digital Library

[30]

Maria Taramigkou, Efthimios Bothos, Konstantinos Christidis, Dimitris Apostolou, and Gregoris Mentzas. 2013. Escape the Bubble: Guided Exploration of Music Preferences for Serendipity and Novelty. In Proceedings of the 7th ACM Conference on Recommender Systems (Hong Kong, China) (RecSys ’13). ACM, New York, NY, USA, 335–338. https://doi.org/10.1145/2507157.2507223

Digital Library

[31]

Cynthia A. Thompson, Mehmet H. Göker, and Pat Langley. 2004. A Personalized System for Conversational Recommendations. Journal of Artificial Intelligence Research 21, 1 (March 2004), 393–428. http://dl.acm.org/citation.cfm?id=1622467.1622479

[32]

Nava Tintarev, Shahin Rostami, and Barry Smyth. 2018. Knowing the unknown: visualising consumption blind-spots in recommender systems. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing. ACM, 1396–1399.

Digital Library

[33]

Saúl Vargas and Pablo Castells. 2013. Exploiting the Diversity of User Preferences for Recommendation. In Proceedings of the 10th Conference on Open Research Areas in Information Retrieval (Lisbon, Portugal) (OAIR ’13). LE CENTRE DE HAUTES ETUDES INTERNATIONALES D’INFORMATIQUE DOCUMENTAIRE, Paris, France, France, 129–136. http://dl.acm.org/citation.cfm?id=2491748.2491776

Digital Library

[34]

Katrien Verbert, Denis Parra, Peter Brusilovsky, and Erik Duval. 2013. Visualizing Recommendations to Support Exploration, Transparency and Controllability. In Proceedings of the 2013 International Conference on Intelligent User Interfaces (Santa Monica, California, USA) (IUI ’13). ACM, New York, NY, USA, 351–362. https://doi.org/10.1145/2449396.2449442

Digital Library

[35]

Rebecca M Warner. 2012. Applied statistics: From bivariate through multivariate techniques. Sage Publications.

[36]

Pontus Wärnestål. 2005. User evaluation of a conversational recommender system. In Proceedings of the 4th IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems. 32–39.

[37]

Wen Wu, Li Chen, and Yu Zhao. 2018. Personalizing recommendation diversity based on user personality. User Modeling and User-Adapted Interaction 28, 3 (01 Aug 2018), 237–276. https://doi.org/10.1007/s11257-018-9205-x

Digital Library

[38]

Longqi Yang, Michael Sobolev, Christina Tsangouri, and Deborah Estrin. 2018. Understanding User Interactions with Podcast Recommendations Delivered via Voice. In Proceedings of the 12th ACM Conference on Recommender Systems (Vancouver, British Columbia, Canada) (RecSys ’18). ACM, New York, NY, USA, 190–194. https://doi.org/10.1145/3240323.3240389

Digital Library

[39]

Jiyong Zhang and Pearl Pu. 2006. A comparative study of compound critique generation in conversational recommender systems. In International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems. Springer, 234–243.

Digital Library

[40]

Mi Zhang and Neil Hurley. 2009. Novel Item Recommendation by User Profile Partitioning. In Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 01(WI-IAT ’09). IEEE Computer Society, Washington, DC, USA, 508–515. https://doi.org/10.1109/WI-IAT.2009.85

Digital Library

[41]

Yuan Cao Zhang, Diarmuid Ó Séaghdha, Daniele Quercia, and Tamas Jambor. 2012. Auralist: Introducing Serendipity into Music Recommendation. In Proceedings of the Fifth ACM International Conference on Web Search and Data Mining (Seattle, Washington, USA) (WSDM ’12). ACM, New York, NY, USA, 13–22. https://doi.org/10.1145/2124295.2124300

Digital Library

[42]

Cai-Nicolas Ziegler, Sean M. McNee, Joseph A. Konstan, and Georg Lausen. 2005. Improving Recommendation Lists Through Topic Diversification. In Proceedings of the 14th International Conference on World Wide Web (Chiba, Japan) (WWW ’05). ACM, New York, NY, USA, 22–32. https://doi.org/10.1145/1060745.1060754

Digital Library

Cited By

Jin YChen LCai WZhao X(2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3631534
Jin YCai WChen LZhang YDoherty GJiang T(2024)Exploring the Design of Generative AI in Supporting Music-based Reminiscence for Older AdultsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642800(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642800
Ma YZiegler J(2024)Investigating meta-intents: user interaction preferences in conversational recommender systemsUser Modeling and User-Adapted Interaction10.1007/s11257-024-09411-3Online publication date: 24-Sep-2024
https://doi.org/10.1007/s11257-024-09411-3
Show More Cited By

Index Terms

Critiquing for Music Exploration in Conversational Recommender Systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Impacts of Personal Characteristics on User Trust in Conversational Recommender Systems
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Conversational recommender systems (CRSs) imitate human advisors to assist users in finding items through conversations and have recently gained increasing attention in domains such as media and e-commerce. Like in human communication, building trust in ...
Latent Linear Critiquing for Conversational Recommender Systems
WWW '20: Proceedings of The Web Conference 2020

Critiquing is a method for conversational recommendation that iteratively adapts recommendations in response to user preference feedback. In this setting, a user is iteratively provided with an item recommendation and attribute description for that item;...
Unifying Recommender Systems and Conversational User Interfaces
CUI '22: Proceedings of the 4th Conference on Conversational User Interfaces

This paper considers unifying research on conversational user interfaces and recommender systems. Studies on conversational user interfaces (CUIs) typically examine how conversations can be facilitated (i.e., optimizing the means). Recommender systems ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '21: Proceedings of the 26th International Conference on Intelligent User Interfaces

April 2021

618 pages

ISBN:9781450380171

DOI:10.1145/3397481

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Hong Kong Baptist University Interdisciplinary Research Clusters Matching Scheme (IRCMS)

Conference

IUI '21

Sponsor:

IUI '21: 26th International Conference on Intelligent User Interfaces

April 14 - 17, 2021

TX, College Station, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
527
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jin YChen LCai WZhao X(2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3631534
Jin YCai WChen LZhang YDoherty GJiang T(2024)Exploring the Design of Generative AI in Supporting Music-based Reminiscence for Older AdultsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642800(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642800
Ma YZiegler J(2024)Investigating meta-intents: user interaction preferences in conversational recommender systemsUser Modeling and User-Adapted Interaction10.1007/s11257-024-09411-3Online publication date: 24-Sep-2024
https://doi.org/10.1007/s11257-024-09411-3
Cai WJin YZhao XChen L(2023)“Listen to Music, Listen to Yourself”: Design of a Conversational Agent to Support Self-Awareness While Listening to MusicProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581427(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581427
Hegiansyah MBaizal Z(2023)Ontology-Based Conversational Recommender System for Smartphone Domain2023 11th International Conference on Information and Communication Technology (ICoICT)10.1109/ICoICT58202.2023.10262611(52-56)Online publication date: 23-Aug-2023
https://doi.org/10.1109/ICoICT58202.2023.10262611
Liang YWillemsen M(2022)Exploring the longitudinal effects of nudging on users’ music genre exploration behavior and listening preferencesProceedings of the 16th ACM Conference on Recommender Systems10.1145/3523227.3546772(3-13)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3523227.3546772
Cai WJin YChen L(2022)Impacts of Personal Characteristics on User Trust in Conversational Recommender SystemsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517471(1-14)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517471
Cai WJin YChen L(2022)Task-Oriented User Evaluation on Critiquing-Based Recommendation ChatbotsIEEE Transactions on Human-Machine Systems10.1109/THMS.2021.313167452:3(354-366)Online publication date: Jun-2022
https://doi.org/10.1109/THMS.2021.3131674
Liang YWillemsen M(2022)Promoting Music Exploration through Personalized Nudging in a Genre Exploration RecommenderInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210806039:7(1495-1518)Online publication date: 21-Aug-2022
https://doi.org/10.1080/10447318.2022.2108060
Jannach DChen L(2022)Conversational recommendationAI Magazine10.1002/aaai.1205943:2(151-163)Online publication date: 23-Jun-2022
https://dl.acm.org/doi/10.1002/aaai.12059
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents