Negative Feedback for Music Personalization
Pages 195 - 200
Abstract
Next-item recommender systems are often trained using only positive feedback with randomly-sampled negative feedback. We show the benefits of using real negative feedback both as inputs into the user sequence and also as negative targets for training a next-song recommender system for internet radio. In particular, using explicit negative samples during training helps reduce training time by ∼ 60% while also improving test accuracy by 6%; adding user skips as additional inputs also can considerably increase user coverage alongside improving accuracy. We test the impact of using a large number of random negative samples to capture a ‘harder’ one and find that the test accuracy increases with more randomly-sampled negatives, but only to a point. Too many random negatives leads to false negatives that limits the lift, which is still lower than if using true negative feedback. We also find that the test accuracy is fairly robust with respect to the proportion of different feedback types, and compare the learned embeddings for different feedback types.
References
[1]
Yoshua Bengio and Jean-SÉbastien Senecal. 2008. Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model. IEEE Transactions on Neural Networks 19, 4 (2008), 713–722. https://doi.org/10.1109/TNN.2007.912312
[2]
Brian Brost, Rishabh Mehrotra, and Tristan Jehan. 2019. The Music Streaming Sessions Dataset. In Proceedings of the 2019 Web Conference. ACM.
[3]
Ricardo Dias and Manuel J Fonseca. 2013. Improving music recommendation in session-based collaborative filtering by using temporal context. In 2013 IEEE 25th international conference on tools with artificial intelligence. IEEE, 783–788. https://doi.org/10.1109/ICTAI.2013.120
[4]
James A Hanley and Barbara J McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology 143, 1 (1982), 29–36.
[5]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 197–206. https://doi.org/10.1109/ICDM.2018.00035
[6]
Francesco Meggetto, Crawford Revie, John Levine, and Yashar Moshfeghi. 2021. On Skipping Behaviour Types in Music Streaming Sessions. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM ’21). Association for Computing Machinery, New York, NY, USA, 3333–3337. https://doi.org/10.1145/3459637.3482123
[7]
Francesco Meggetto, Crawford Revie, John Levine, and Yashar Moshfeghi. 2023. Why People Skip Music? On Predicting Music Skips using Deep Reinforcement Learning. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (, Austin, TX, USA, ) (CHIIR ’23). Association for Computing Machinery, New York, NY, USA, 95–106. https://doi.org/10.1145/3576840.3578312
[8]
M. Jeffrey Mei, Oliver Bembom, and Andreas Ehmann. 2023. Station and Track Attribute-Aware Music Personalization. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 1031–1035. https://doi.org/10.1145/3604915.3610239
[9]
Elisa Mena-Maldonado, Rocío Cañamares, Pablo Castells, Yongli Ren, and Mark Sanderson. 2020. Agreement and Disagreement between True and False-Positive Metrics in Recommender Systems Evaluation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, China) (SIGIR ’20). Association for Computing Machinery, New York, NY, USA, 841–850. https://doi.org/10.1145/3397271.3401096
[10]
Aleksandr Vladimirovich Petrov and Craig Macdonald. 2023. GSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 116–128. https://doi.org/10.1145/3604915.3608783
[11]
Markus Schedl and Dominik Schnitzer. 2014. Location-aware music artist recommendation. In MultiMedia Modeling: 20th Anniversary International Conference, MMM 2014, Dublin, Ireland, January 6-10, 2014, Proceedings, Part II 20. Springer, 205–213.
[12]
Pavan Seshadri and Peter Knees. 2023. Leveraging Negative Signals with Self-Attention for Sequential Music Recommendation. arXiv preprint arXiv:2309.11623 (2023).
[13]
Sasha Stoikov and Hongyi Wen. 2021. Evaluating Music Recommendations with Binary Feedback for Multiple Stakeholders. In Proceedings of the 1st Workshop on Multi-Objective Recommender Systems (MORS 2021) co-located with 15th ACM Conference on Recommender Systems (RecSys 2021)(CEUR Workshop Proceedings, Vol. 2959). CEUR-WS.org. https://ceur-ws.org/Vol-2959/paper9.pdf
[14]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (Beijing, China) (CIKM ’19). Association for Computing Machinery, New York, NY, USA, 1441–1450. https://doi.org/10.1145/3357384.3357895
[15]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017). https://doi.org/10.48550/arXiv.1706.03762
[16]
Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-Scale Commodity Embedding for E-Commerce Recommendation in Alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (London, United Kingdom) (KDD ’18). Association for Computing Machinery, New York, NY, USA, 839–848. https://doi.org/10.1145/3219819.3219869
[17]
Xinxi Wang and Ye Wang. 2014. Improving content-based and hybrid music recommendation using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia. 627–636. https://doi.org/10.1145/2647868.2654940
[18]
Yueqi Wang, Yoni Halpern, Shuo Chang, Jingchen Feng, Elaine Ya Le, Longfei Li, Xujian Liang, Min-Cheng Huang, Shane Li, Alex Beutel, Yaping Zhang, and Shuchao Bi. 2023. Learning from Negative User Feedback and Measuring Responsiveness for Sequential Recommenders. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 1049–1053. https://doi.org/10.1145/3604915.3610244
[19]
Hongyi Wen, Longqi Yang, and Deborah Estrin. 2019. Leveraging post-click feedback for content recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys ’19). Association for Computing Machinery, New York, NY, USA, 278–286. https://doi.org/10.1145/3298689.3347037
[20]
Jason Weston, Samy Bengio, and Nicolas Usunier. 2011. Wsabie: Scaling Up To Large Vocabulary Image Annotation. In Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI.
[21]
Timo Wilm, Philipp Normann, Sophie Baumeister, and Paul-Vincent Kobow. 2023. Scaling Session-Based Transformer Recommendations Using Optimized Negative Sampling and Loss Functions(RecSys ’23). Association for Computing Machinery, New York, NY, USA, 1023–1026. https://doi.org/10.1145/3604915.3610236
[22]
Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, Tianyu Qiu, and Xiangnan He. 2023. On the Effectiveness of Sampled Softmax Loss for Item Recommendation. ACM Trans. Inf. Syst. (dec 2023). https://doi.org/10.1145/3637061 Just Accepted.
[23]
Yuhan Zhao, Rui Chen, Riwei Lai, Qilong Han, Hongtao Song, and Li Chen. 2023. Augmented Negative Sampling for Collaborative Filtering. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’23). Association for Computing Machinery, New York, NY, USA, 256–266. https://doi.org/10.1145/3604915.3608811
Index Terms
- Negative Feedback for Music Personalization
Recommendations
When humans and computers induce social stress through negative feedback: Effects on performance and subjective state
AbstractPeople increasingly work with autonomous systems, which progressively take over functions previously performed exclusively by humans. This may lead to situations in which automated agents give negative performance feedback, which ...
Highlights- Negative performance feedback from both human and computer did not impair subsequent performance on a wide range of tasks.
Enhancing Sequential Music Recommendation with Negative Feedback-informed Contrastive Learning
RecSys '24: Proceedings of the 18th ACM Conference on Recommender SystemsModern music streaming services are heavily based on recommendation engines to serve content to users. Sequential recommendation—continuously providing new items within a single session in a contextually coherent manner—has been an emerging topic in ...
A study of methods for negative relevance feedback
SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrievalNegative relevance feedback is a special case of relevance feedback where we do not have any positive example; this often happens when the topic is difficult and the search results are poor. Although in principle any standard relevance feedback ...
Comments
Information & Contributors
Information
Published In
June 2024
338 pages
ISBN:9798400704338
DOI:10.1145/3627043
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 June 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
UMAP '24: 32nd ACM Conference on User Modeling, Adaptation and Personalization
July 1 - 4, 2024
Cagliari, Italy
Acceptance Rates
Overall Acceptance Rate 162 of 633 submissions, 26%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 53Total Downloads
- Downloads (Last 12 months)53
- Downloads (Last 6 weeks)12
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format