short-paper

Multi-armed recommender system bandit ensembles

Authors:

Rocío Cañamares,

Marcos Redondo,

Pablo CastellsAuthors Info & Claims

RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

Pages 432 - 436

https://doi.org/10.1145/3298689.3346984

Published: 10 September 2019 Publication History

Abstract

It has long been found that well-configured recommender system ensembles can achieve better effectiveness than the combined systems separately. Sophisticated approaches have been developed to automatically optimize the ensembles' configuration to maximize their performance gains. However most work in this area has targeted simplified scenarios where algorithms are tested and compared on a single non-interactive run. In this paper we consider a more realistic perspective bearing in mind the cyclic nature of the recommendation task, where a large part of the system's input is collected from the reaction of users to the recommendations they are delivered. The cyclic process provides the opportunity for ensembles to observe and learn about the effectiveness of the combined algorithms, and improve the ensemble configuration progressively.

In this paper we explore the adaptation of a multi-armed bandit approach to achieve this, by representing the combined systems as arms, and the ensemble as a bandit that at each step selects an arm to produce the next round of recommendations. We report experiments showing the effectiveness of this approach compared to ensembles that lack the iterative perspective. Along the way, we find illustrative pitfall examples that can result from common, single-shot offline evaluation setups.

References

[1]

G. Adomavicius and A. Tuzhilin (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17, 6 (June 2005), 734--749.

Digital Library

[2]

F. Aksel and A. Birtürk (2010). An Adaptive Hybrid Recommender System that Learns Domain Dynamics. In International Workshop on Handling Concept Drift in Adaptive Information Systems: Importance, Challenges and Solutions (HaCDAIS-2010) at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2010). Barcelona, Spain, 49--56.

[3]

A. Bar, L. Rokach, G. Shani, B. Shapira and A. Schclar (2013). Improving Simple Collaborative Filtering Models Using Ensemble Methods. In 11<sup>th</sup> International Workshop on Multiple Classifier Systems (MCS 2013). Nanjing, China, 1--12.

[4]

R. Bell, Y. Koren and C. Volinsky (2007). The bellkor solution to the Netflix prize. KorBell Team's Report to Netflix (2007).

[5]

B. Brodén, M. Hammar, B. Nilson and D. Paraschakis (2018). Ensemble Recommendations via Thompson Sampling: an Experimental Study within e-Commerce. In Proceedings of 23<sup>rd</sup> International Conference on Intelligent User Interfaces (IUI 2018). Tokyo Japan, 19--29.

Digital Library

[6]

R. Burke (2002). Hybrid Recommender Systems: Survey and Experiments. User Modeling and User-Adapted Interaction, 12, 4 (November 2002). Kluwer Academic Publishers Hingham, MA, USA, 331--370.

Digital Library

[7]

R. Cañamares and P. Castells (2017). A Probabilistic Reformulation of Memory-Based Collaborative Filtering - Implications on Popularity Biases. In Proceeding of the 40<sup>th</sup> Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2017). ACM, New York, USA, 215--224.

Digital Library

[8]

O. Chapelle and L. Li (2011). An empirical evaluation of Thompson Sampling. In Proceedings of Neural Information Processing Systems (NIPS 2011). Curran Associates, Inc., Red Hook, NY, USA, 2249--2257.

Digital Library

[9]

S. Dooms (2013). Dynamic generation of personalized hybrid recommender systems. In Proceedings of the 7<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2013). Hong Kong, China, 443--446.

Digital Library

[10]

A. Gilotte, C. Calauzènes, T. Nedelec, A. Abraham and S. Dollé (2018). Offline A/B Testing for Recommender Systems. In Proceedings of the 11<sup>th</sup> ACM International Conference on Web Search and Data Mining (WSDM 2018). ACM, New York, NY, USA, 198--206.

Digital Library

[11]

D. Hill, H. Nassif, Y. Liu, A. Iyer and S. Vishwanathan (2017). An Efficient Bandit Algorithm for Realtime Multivariate Optimization. In Proceedings of the 23<sup>rd</sup> ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2017). Halifax, NS, Canada, 1813--1821.

Digital Library

[12]

R. Kohavi, R. Longbotham, D. Sommerfield and R. Henne (2009). Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery, 18, 1 (February 2009), 140--181.

Digital Library

[13]

Y. Hu, Y. Koren and C. Volinsky (2008). Collaborative Filtering for Implicit Feedback Datasets. In Proceedings of the 8<sup>th</sup> IEEE International Conference on Data Mining (ICDM 2008). IEEE Computer Society, Washington, DC, USA, 15--19.

Digital Library

[14]

J. Kawale, H. H. Bui, B. Kveton, L. Tran-Thanh and S. Chawla (2015). Efficient Thompson Sampling for Online Matrix-Factorization Recommendation. In Proceedings of Neural Information Processing Systems (NIPS 2015). Curran Associates, Inc., Red Hook, NY, USA, 1297--1305.

Digital Library

[15]

P. Kouki, S. Fakhraei, J. Foulds, M. Eirinaki and L. Getoor (2015). HyPER: A Flexible and Extensible Probabilistic Framework for Hybrid Recommender Systems. In Proceedings of the 9<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2015). ACM, New York, NY, USA, 99--106.

Digital Library

[16]

L. Li, W. Chu, J. Langford and R. Schapire (2010). A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19<sup>th</sup> International Conference on World Wide Web (WWW 2010). ACM, New York, NY, USA, 661--670.

Digital Library

[17]

S. Li, A. Karatzoglou, and C. Gentile (2016). Collaborative Filtering Bandits. In Proceedings of the 39<sup>th</sup> International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016). ACM New York, NY, USA, 539--548.

Digital Library

[18]

F. M. Maxwell and J. A. Konstan (2015). The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 5, 4 (December 2015).

Digital Library

[19]

X. Ning, C. Desrosiers and G. Karypis (2015). A Comprehensive Survey of Neighborhood-Based Recommendation Methods. In F. Ricci, L. Rokach and B. Shapira (Eds.), Recommender Systems Handbook (2<sup>nd</sup> ed.). Springer, New York, NY, USA, 37--76.

[20]

K. Pang, M. Dong, Y. Wu and T. Hospedales (2018). Dynamic Ensemble Active Learning: A Non-Stationary Bandit with Expert Advice. In Proceedings of the 24th International Conference on Pattern Recognition (ICPR 2018). IEEE Computer Society, Washington, DC, USA, 2269--2276.

[21]

V. Perchet, P. Rigollet, S, Chassang and E. Snowberg (2016). Batched Bandit Problems. Annals of Statistics, 44, 2 (April 2016), 660--681.

[22]

D. Siroker and P. Koomen (2015). A/B testing: the most powerful way to turn clicks into customers. John Wiley & Sons Inc, Hoboken, NJ, USA, 2015.

Digital Library

[23]

R. Sutton and A. Barto (2018). Reinforcement Learning: An Introduction (2<sup>nd</sup> ed.). MIT Press, Cambridge, MA, USA, 2018.

Digital Library

[24]

L. Tang, Y. Jiang, L. Li and T. Li (2014). Ensemble contextual bandits for personalized recommendation. In Proceedings of the 8<sup>th</sup> ACM Conference on Recommender Systems (RecSys 2014). Foster City, CA, USA, 73--80.

Digital Library

[25]

Q. Wang, C. Zeng, W. Zhou, T. Li, S. S. Iyengar, L. Shwartz and G. Grabarnik (2019). Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms. IEEE Transactions on Knowledge and Data Engineering, 31, 8 (August 2019), 1569--1580.

Digital Library

[26]

X. Zhao, W. Zhang and J. Wang (2013). Interactive Collaborative Filtering. In Proceedings of the 22<sup>nd</sup> ACM International Conference on Information and Knowledge Management (CIKM 2013). ACM, New York, NY, USA, 1411--1420.

Digital Library

Cited By

Rajapakse DLeith D(2024)User Cold-Start Learning in Recommender Systems using Monte Carlo Tree SearchACM Transactions on Recommender Systems10.1145/36180023:1(1-23)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3618002
Wegmeth L(2023)Improving Recommender Systems Through the Automation of Design DecisionsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608877(1332-1338)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608877
Dekel LLeybovich IZilberman PPuzis R(2023)MABAT: A Multi-Armed Bandit Approach for Threat-HuntingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2022.321501018(477-490)Online publication date: 2023
https://doi.org/10.1109/TIFS.2022.3215010
Show More Cited By

Index Terms

Multi-armed recommender system bandit ensembles
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
    2. Machine learning algorithms
      1. Ensemble methods
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

A simple multi-armed nearest-neighbor bandit for interactive recommendation
RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

The cyclic nature of the recommendation task is being increasingly taken into account in recommender systems research. In this line, framing interactive recommendation as a genuine reinforcement learning problem, multi-armed bandit approaches have been ...
A Multi-Armed Bandit Model Selection for Cold-Start User Recommendation
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

How can we effectively recommend items to a user about whom we have no information? This is the problem we focus on in this paper, known as the cold-start problem. In most existing works, the cold-start problem is handled through the use of many kinds ...
Group recommendations via multi-armed bandits
WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

We study recommendations for persistent groups that repeatedly engage in a joint activity. We approach this as a multi-arm bandit problem. We design a recommendation policy and show it has logarithmic regret. Our analysis also shows that regret depends ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

September 2019

635 pages

ISBN:9781450362436

DOI:10.1145/3298689

General Chairs:
Toine Bogers
Aalborg University Copenhagen, Denmark
,
Alan Said
University of Gothenburg, Sweden
,
Program Chairs:
Peter Brusilovsky
University of Pittsburgh
,
Domonkos Tikk
Gravity R&D, Hungary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 September 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Ministerio de Ciencia, Innovación y Universidades

Conference

RecSys '19

RecSys '19: Thirteenth ACM Conference on Recommender Systems

September 16 - 20, 2019

Copenhagen, Denmark

Acceptance Rates

RecSys '19 Paper Acceptance Rate 36 of 189 submissions, 19%;

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
778
Total Downloads

Downloads (Last 12 months)75
Downloads (Last 6 weeks)5

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rajapakse DLeith D(2024)User Cold-Start Learning in Recommender Systems using Monte Carlo Tree SearchACM Transactions on Recommender Systems10.1145/36180023:1(1-23)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3618002
Wegmeth L(2023)Improving Recommender Systems Through the Automation of Design DecisionsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608877(1332-1338)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608877
Dekel LLeybovich IZilberman PPuzis R(2023)MABAT: A Multi-Armed Bandit Approach for Threat-HuntingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2022.321501018(477-490)Online publication date: 2023
https://doi.org/10.1109/TIFS.2022.3215010
Miyake YMine T(2023)Contextual and Nonstationary Multi-armed Bandits Using the Linear Gaussian State Space Model for the Meta-Recommender System2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394517(3138-3145)Online publication date: 1-Oct-2023
https://doi.org/10.1109/SMC53992.2023.10394517
Zuin GParreiras LMelo LBarros GLomeu HMelo BMarini WLott DDe Souza M(2023)An Ensemble Approach for Inconsistency Detection in Medical Bills: A Case Study2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)10.1109/CBMS58004.2023.00281(573-578)Online publication date: Jun-2023
https://doi.org/10.1109/CBMS58004.2023.00281
Sethuraman SUma Maheswari GThombre SKumar SPatel VRamanan S(2023)ENCODE: Ensemble Contextual Bandits in Big Data Settings - A Case Study in E-Commerce Dynamic Pricing2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386412(5372-5381)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386412
Wang XOunis IMacdonald C(2022)BanditProp: Bandit Selection of Review Properties for Effective RecommendationACM Transactions on the Web10.1145/353285916:4(1-19)Online publication date: 16-Nov-2022
https://dl.acm.org/doi/10.1145/3532859
Rajapakse DLeith D(2022)Fast and Accurate User Cold-Start Learning Using Monte Carlo Tree SearchProceedings of the 16th ACM Conference on Recommender Systems10.1145/3523227.3546786(350-359)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3523227.3546786
Gulla JSvendsen RZhang LStenbom AFrøland J(2021)Recommending news in traditional media companiesAI Magazine10.1609/aimag.v42i3.1814642:3(55-69)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1609/aimag.v42i3.18146
Zavrel JGrotov AMitnik J(2021)Building a Platform for Ensemble-based Personalized Research Literature Recommendations for AI and Data Science at Zeta AlphaProceedings of the 15th ACM Conference on Recommender Systems10.1145/3460231.3474619(536-537)Online publication date: 13-Sep-2021
https://dl.acm.org/doi/10.1145/3460231.3474619
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents