It's Not You, It's Me: The Impact of Choice Models and Ranking Strategies on Gender Imbalance in Music Recommendation
Pages 884 - 889
Abstract
As recommender systems are prone to various biases, mitigation approaches are needed to ensure that recommendations are fair to various stakeholders. One particular concern in music recommendation is artist gender fairness. Recent work has shown that the gender imbalance in the sector translates to the output of music recommender systems, creating a feedback loop that can reinforce gender biases over time.
In this work, we examine that feedback loop to study whether algorithmic strategies or user behavior are a greater contributor to ongoing improvement (or loss) in fairness as models are repeatedly re-trained on new user feedback data. We simulate user interaction and re-training to investigate the effects of ranking strategies and user choice models on gender fairness metrics. We find re-ranking strategies have a greater effect than user choice models on recommendation fairness over time.
References
[1]
Christine Bauer and Andres Ferraro. 2023. Strategies for Mitigating Artist Gender Bias in Music Recommendation: a Simulation Study. In Proceedings of the Music Recommender Systems Workshop (Singapore) (MuRS 2023). Zenodo, 5 pages. https://doi.org/10.5281/zenodo.8372477
[2]
Òscar Celma and Pedro Cano. 2008. From Hits to Niches? Or How Popular Artists Can Bias Music Recommendation and Discovery. In Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition (Las Vegas, NV, USA) (NETFLIX ’08). Association for Computing Machinery, New York, NY, USA, Article 5, 8 pages. https://doi.org/10.1145/1722149.1722154
[3]
Allison J. B. Chaney, Brandon M. Stewart, and Barbara E. Engelhardt. 2018. How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility. In Proceedings of the 12th ACM Conference on Recommender Systems (Vancouver, BC, Canada) (RecSys ’18). Association for Computing Machinery, New York, NY, USA, 224–232. https://doi.org/10.1145/3240323.3240370
[4]
Andrew Collins, Dominika Tkaczyk, Akiko Aizawa, and Joeran Beel. 2018. Position Bias in Recommender Systems for Digital Libraries. In Transforming Digital Worlds (Sheffield, UK) (iConference 2018). Springer, Cham, Switzerland, 335–344. https://doi.org/10.1007/978-3-319-78105-1_37
[5]
Karlijn Dinnissen and Christine Bauer. 2023. Amplifying Artists’ Voices: Item Provider Perspectives on Influence and Fairness of Music Streaming Platforms. In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization (Limassol, Cyprus) (UMAP ’23). Association for Computing Machinery, New York, NY, USA, 238–249. https://doi.org/10.1145/3565472.3592960
[6]
Michael D. Ekstrand. 2020. LensKit for Python: Next-Generation Software for Recommender Systems Experiments. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, New York, NY, USA, 2999–3006. https://doi.org/10.1145/3340531.3412778
[7]
Michael D. Ekstrand, Ben Carterette, and Fernando Diaz. 2024. Distributionally-Informed Recommender System Evaluation. ACM Transactions on Recommender Systems 2, 1, Article 6 (March 2024), 27 pages. https://doi.org/10.1145/3613455
[8]
Michael D. Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2022. Fairness in Information Access Systems. Foundations and Trends® in Information Retrieval 16, 1–2 (2022), 196 pages. https://doi.org/10.1561/1500000079
[9]
Michael D. Ekstrand and Daniel Kluver. 2021. Exploring Author Gender in Book Rating and Recommendation. User Modeling and User-Adapted Interaction 31, 3 (2021), 377–420. https://doi.org/10.1007/s11257-020-09284-2
[10]
Michael D. Ekstrand and Vaibhav Mahant. 2017. Sturgeon and the Cool Kids: Problems with Top-N Recommender Evaluation. In Proceedings of the 30th Florida Artificial Intelligence Research Society Conference (Marco Island, FL, USA) (FLAIRS 2017). AAAI Press, Washington, DC, USA, 639–644. https://aaai.org/papers/639-flairs-2017-15534/
[11]
Michael D Ekstrand, Graham McDonald, Amifa Raj, and Isaac Johnson. 2022. Overview of the TREC 2021 Fair Ranking Track. In The Thirtieth Text REtrieval Conference (TREC 2021) Proceedings. https://trec.nist.gov/pubs/trec30/papers/Overview-F.pdf
[12]
Mehdi Elahi, Danial Khosh Kholgh, Mohammad Sina Kiarostami, Sorush Saghari, Shiva Parsa Rad, and Marko Tkalčič. 2021. Investigating the Impact of Recommender Systems on User-based and Item-based Popularity Bias. Information Processing & Management 58, 5, Article 102655 (Sept. 2021), 15 pages. https://doi.org/10.1016/j.ipm.2021.102655
[13]
Avriel Epps-Darling, Romain Takeo Bouyer, and Henriette Cramer. 2020. Artist Gender Representation in Music Streaming. In Proceedings of the 21st International Society for Music Information Retrieval Conference (Montréal, Canada) (ISMIR 2020). ISMIR, 248–254. https://doi.org/10.5281/zenodo.4245416
[14]
Francesco Fabbri, Maria Luisa Croci, Francesco Bonchi, and Carlos Castillo. 2022. Exposure Inequality in People Recommender Systems: The Long-Term Effects. Proceedings of the International AAAI Conference on Web and Social Media 16, 1 (May 2022), 194–204. https://doi.org/10.1609/icwsm.v16i1.19284
[15]
Andres Ferraro, Dmitry Bogdanov, Xavier Serra, and Jason Yoon. 2019. Artist and Style Exposure Bias in Collaborative Filtering Based Music Recommendations. In Proceedings of the 1st Workshop on Designing Human-Centric Music Information Research Systems (Delft, The Netherlands) (wsHCMIR ’19). 8–10. https://doi.org/10.48550/arXiv.1911.04827
[16]
Andres Ferraro, Xavier Serra, and Christine Bauer. 2021. Break the Loop: Gender Imbalance in Music Recommenders. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (Canberra, ACT, Australia) (CHIIR ’21). Association for Computing Machinery, New York, NY, USA, 249–254. https://doi.org/10.1145/3406522.3446033
[17]
Andres Ferraro, Xavier Serra, and Christine Bauer. 2021. What Is Fair? Exploring the Artists’ Perspective on the Fairness of Music Streaming Platforms. In Human-Computer Interaction – INTERACT 2021: 18th IFIP TC 13 International Conference (Bari, Italy) (INTERACT ’21, Vol. 12933). Springer, Cham, Switzerland, 562–584. https://doi.org/10.1007/978-3-030-85616-8_33
[18]
Naieme Hazrati and Francesco Ricci. 2022. Recommender Systems Effect on the Evolution of Users’ Choices Distribution. Information Processing & Management 59, 1, Article 102766 (2022), 18 pages. https://doi.org/10.1016/j.ipm.2021.102766
[19]
Naieme Hazrati and Francesco Ricci. 2024. Choice Models and Recommender Systems Effects on Users’ Choices. User Modeling and User-Adapted Interaction 34, 1 (2024), 109–145. https://doi.org/10.1007/s11257-023-09366-x
[20]
Karla Hernandez, Stacy L. Smith, Marc Choueiti, and Katherine Pieper. 2022. Inclusion in the Recording Studio?: Gender and Race/Ethnicity of Artists, Songwriters & Producers across 1,000 Popular Songs from 2012–2021. Technical Report. Annenberg Inclusion Initiative. https://assets.uscannenberg.org/docs/aii-inclusion-recording-studio-20220331.pdf
[21]
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. In 2008 Eighth IEEE International Conference on Data Mining (Pisa, Italy) (ICDM 2008). IEEE, New York, NY, USA, 263–272. https://doi.org/10.1109/ICDM.2008.22
[22]
IFPI. 2024. IFPI Global Music Report 2024: State of the Industry. Technical Report. IFPI, London, UK. https://ifpi-website-cms.s3.eu-west-2.amazonaws.com/IFPI_GMR_2024_State_of_the_Industry_db92a1c9c1.pdf
[23]
Dietmar Jannach, Lukas Lerche, Iman Kamehkhosh, and Michael Jugovac. 2015. What Recommenders Recommend: an Analysis of Recommendation Biases and Possible Countermeasures. User Modeling and User-Adapted Interaction 25, 5 (2015), 427–491. https://doi.org/10.1007/s11257-015-9165-3
[24]
Dominik Kowald and Emanuel Lacic. 2022. Popularity Bias in Collaborative Filtering-Based Multimedia Recommender Systems. In Advances in Bias and Fairness in Information Retrieval(BIAS 2022, Vol. 1610). Springer, Cham, Switzerland, 1–11. https://doi.org/10.1007/978-3-031-09316-6_1
[25]
Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020. Feedback Loop and Bias Amplification in Recommender Systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, New York, NY, USA, 2145–2148. https://doi.org/10.1145/3340531.3412152
[26]
Shira Mitchell, Eric Potash, Solon Barocas, Alexander D’Amour, and Kristian Lum. 2021. Algorithmic Fairness: Choices, Assumptions, and Definitions. Annual Review of Statistics and Its Application 8 (Nov. 2021), 141–163. https://doi.org/10.1146/annurev-statistics-042720-125902
[27]
Amifa Raj and Michael Ekstrand. 2023. Unified Browsing Models for Linear and Grid Layouts. CoRRarXiv:2310.12524 (2023), 7 pages. https://doi.org/10.48550/arXiv.2310.12524
[28]
Amifa Raj and Michael D Ekstrand. 2022. Measuring Fairness in Ranked Results: An Analytical and Empirical Comparison. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 726–736. https://doi.org/10.1145/3477495.3532018
[29]
Amifa Raj and Michael D. Ekstrand. 2024. Towards Optimizing Ranking in Grid-Layout for Provider-Side Fairness. In Advances in Information Retrieval (Glasgow, UK) (ECIR 2024, Vol. 14612). Springer, Cham, Switzerland, 90–105. https://doi.org/10.1007/978-3-031-56069-9_7
[30]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal, Quebec, Canada) (UAI ’09). AUAI Press, Arlington, VA, USA, 452–461. https://doi.org/10.5555/1795114.1795167
[31]
Piotr Sapiezynski, Wesley Zeng, Ronald E Robertson, Alan Mislove, and Christo Wilson. 2019. Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists. In Companion Proceedings of The 2019 World Wide Web Conference (San Francisco, CA, USA) (WWW ’19 Companion). Association for Computing Machinery, New York, NY, USA, 553–562. https://doi.org/10.1145/3308560.3317595
[32]
Markus Schedl, Stefan Brandl, Oleg Lesota, Emilia Parada-Cabaleiro, David Penz, and Navid Rekabsaz. 2022. LFM-2b: A Dataset of Enriched Music Listening Events for Recommender Systems Research and Fairness Analysis. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval (Regensburg, Germany) (CHIIR ’22). Association for Computing Machinery, New York, NY, USA, 337–341. https://doi.org/10.1145/3498366.3505791
[33]
Markus Schedl, Peter Knees, Brian McFee, and Dmitry Bogdanov. 2022. Music Recommendation Systems: Techniques, Use Cases, and Challenges. In Recommender Systems Handbook (3rd ed.), Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, New York, NY, USA, 927–971. https://doi.org/10.1007/978-1-0716-2197-4_24
[34]
Stacy L. Smith, Katherine Pieper, Karla Hernandez, and Sam Wheeler. 2024. Inclusion in the Recording Studio?: Gender and Race/Ethnicity of Artists, Songwriters & Producers across 1,200 Popular Songs from 2012–2023. Technical Report. Annenberg Inclusion Initiative. https://assets.uscannenberg.org/docs/aii-inclusion-recording-studio-20240130.pdf
[35]
Nasim Sonboli, Farzad Eskandanian, Robin Burke, Weiwen Liu, and Bamshad Mobasher. 2020. Opportunistic Multi-aspect Fairness through Personalized Re-ranking. In UMAP ’20. Association for Computing Machinery, New York, NY, USA, 239–247. https://doi.org/10.1145/3340631.3394846
[36]
Nasim Sonboli, Farzad Eskandanian, Robin Burke, Weiwen Liu, and Bamshad Mobasher. 2020. Opportunistic Multi-aspect Fairness through Personalized Re-ranking. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (Genoa, Italy) (UMAP ’20). Association for Computing Machinery, New York, NY, USA, 239–247. https://doi.org/10.1145/3340631.3394846
[37]
Yixue Wang and Emőke-Ágnes Horvát. 2019. Gender Differences in the Global Music Industry: Evidence from MusicBrainz and The Echo Nest. Proceedings of the International AAAI Conference on Web and Social Media 13, 01 (July 2019), 517–526. https://doi.org/10.1609/icwsm.v13i01.3249
[38]
Yifan Wang, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. 2023. A Survey on the Fairness of Recommender Systems. ACM Transactions on Information Systems 41, 3, Article 52 (Feb. 2023), 43 pages. https://doi.org/10.1145/3547333
[39]
Jada Watson. 2020. Programming Inequality: Gender Representation on Canadian Country Radio (2005–2019). In Proceedings of the 21st International Society for Music Information Retrieval Conference (Montréal, Canada) (ISMIR 2020). ISMIR, 392–399. https://doi.org/10.5281/zenodo.4245452
[40]
Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017. FA*IR: A Fair Top-k Ranking Algorithm. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (Singapore) (CIKM ’17). Association for Computing Machinery, New York, NY, USA, 1569–1578. https://doi.org/10.1145/3132847.3132938
[41]
Jingjng Zhang, Gediminas Adomavicius, Alok Gupta, and Wolfgang Ketter. 2020. Consumption and Performance: Understanding Longitudinal Dynamics of Recommender Systems via an Agent-Based Simulation Framework. Information Systems Research 31, 1 (2020), 76–101. https://doi.org/10.1287/isre.2019.0876
Index Terms
- It's Not You, It's Me: The Impact of Choice Models and Ranking Strategies on Gender Imbalance in Music Recommendation
Recommendations
Break the Loop: Gender Imbalance in Music Recommenders
CHIIR '21: Proceedings of the 2021 Conference on Human Information Interaction and RetrievalAs recommender systems play an important role in everyday life, there is an increasing pressure that such systems are fair. Besides serving diverse groups of users, recommenders need to represent and serve item providers fairly as well. In interviews ...
Investigating gender fairness of recommendation algorithms in the music domain
AbstractAlthough recommender systems (RSs) play a crucial role in our society, previous studies have revealed that the performance of RSs may considerably differ between groups of individuals with different characteristics or from different ...
Highlights- Novel large-scale real-world dataset of music listening records.
- Debiasing ...
Girls Rocking the Code: Gender-dependent Stereotypes, Engagement & Comprehension in Music Programming
ICSE-SEET '24: Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and TrainingOne of the greatest challenges in early programming education is to achieve learning success while also creating initial interest. This is particularly difficult for girls, who continue to be affected by persistent societal stereotypes and negative ...
Comments
Information & Contributors
Information
Published In
October 2024
1438 pages
ISBN:9798400705052
DOI:10.1145/3640457
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.
Sponsors
- SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
- SIGAI: ACM Special Interest Group on Artificial Intelligence
- SIGKDD: ACM Special Interest Group on Knowledge Discovery in Data
- SIGIR: ACM Special Interest Group on Information Retrieval
- SIGCHI: ACM Special Interest Group on Computer-Human Interaction
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 October 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Funding Sources
- U.S. National Science Foundation
- Salzburger Landesregierung
Conference
Acceptance Rates
Overall Acceptance Rate 254 of 1,295 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 285Total Downloads
- Downloads (Last 12 months)285
- Downloads (Last 6 weeks)76
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in