research-article

Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems

Authors:

Sinno Jialin Pan,

Anxiang ZengAuthors Info & Claims

RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems

Pages 411 - 421

https://doi.org/10.1145/3460231.3474239

Published: 13 September 2021 Publication History

Abstract

Recommender Systems (RSs) in real-world applications often deal with billions of user interactions daily. To capture the most recent trends effectively, it is common to update the model incrementally using only the newly arrived data. However, this may impede the model’s ability to retain long-term information due to the potential overfitting and forgetting issues. To address this problem, we propose a novel Adaptive Sequential Model Generation (ASMG) framework, which generates a better serving model from a sequence of historical models via a meta generator. For the design of the meta generator, we propose to employ Gated Recurrent Units (GRUs) to leverage its ability to capture the long-term dependencies. We further introduce some novel strategies to apply together with the GRU meta generator, which not only improve its computational efficiency but also enable more accurate sequential modeling. By instantiating the model-agnostic framework on a general deep learning-based RS model, we demonstrate that our method achieves state-of-the-art performance on three public datasets and one industrial dataset.

Supplementary Material

MP4 File (recsys_video.mp4)

Presentation video of paper "Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems", published at RecSys 2021.

Download
23.41 MB

References

[1]

Chen Chen, Hongzhi Yin, Junjie Yao, and Bin Cui. 2013. Terec: A temporal recommender system over tweet stream. Proceedings of the VLDB Endowment 6, 12 (2013), 1254–1257.

Digital Library

[2]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, 2016. Wide & Deep Learning for Recommender Systems. In DLRS@ RecSys.

[3]

Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. In NIPS 2014 Workshop on Deep Learning, December 2014.

[4]

Robin Devooght, Nicolas Kourtellis, and Amin Mantrach. 2015. Dynamic matrix factorization with priors on unknown values. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 189–198.

Digital Library

[5]

Ernesto Diaz-Aviles, Lucas Drumond, Lars Schmidt-Thieme, and Wolfgang Nejdl. 2012. Real-time top-n recommendation in social streams. In Proceedings of the sixth ACM conference on Recommender systems. 59–66.

Digital Library

[6]

Erzsébet Frigó, Róbert Pálovics, Domokos Kelen, Levente Kocsis, and András Benczúr. 2017. Online ranking prediction in non-stationary environments. (2017).

[7]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1725–1731.

[8]

Lei Guo, Hongzhi Yin, Qinyong Wang, Tong Chen, Alexander Zhou, and Nguyen Quoc Viet Hung. 2019. Streaming session-based recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1569–1577.

Digital Library

[9]

Michael Jugovac, Dietmar Jannach, and Mozhgan Karimi. 2018. Streamingrec: a framework for benchmarking stream-based news recommenders. In Proceedings of the 12th ACM Conference on Recommender Systems. 269–273.

Digital Library

[10]

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114, 13(2017), 3521–3526.

[11]

Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7765–7773.

[12]

Fei Mi and Boi Faltings. 2020. Memory Augmented Neural Model for Incremental Session-based Recommendation. arXiv preprint arXiv:2005.01573(2020).

[13]

Fei Mi, Xiaoyu Lin, and Boi Faltings. 2020. Ader: Adaptively distilled exemplar replay towards continual learning for session-based recommendation. In Fourteenth ACM Conference on Recommender Systems. 408–413.

Digital Library

[14]

Manos Papagelis, Ioannis Rousidis, Dimitris Plexousakis, and Elias Theoharopoulos. 2005. Incremental collaborative filtering for highly-scalable recommendation algorithms. In International Symposium on Methodologies for Intelligent Systems. Springer, 553–561.

Digital Library

[15]

Ruihong Qiu, Hongzhi Yin, Zi Huang, and Tong Chen. 2020. Gag: Global attributed graph neural network for streaming session-based recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 669–678.

Digital Library

[16]

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2001–2010.

[17]

Steffen Rendle and Lars Schmidt-Thieme. 2008. Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In Proceedings of the 2008 ACM conference on Recommender systems. 251–258.

Digital Library

[18]

Anthony Robins. 1995. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science 7, 2 (1995), 123–146.

[19]

Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671(2016).

[20]

Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 255–262.

Digital Library

[21]

Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Advances in neural information processing systems. 2990–2999.

[22]

João Vinagre, Alípio Mário Jorge, and João Gama. 2014. Fast incremental matrix factorization for recommendation with positive-only feedback. In International Conference on User Modeling, Adaptation, and Personalization. Springer, 459–470.

[23]

JianGuo Wang, Joshua Zhexue Huang, Dingming Wu, Jiafeng Guo, and Yanyan Lan. 2016. An incremental model on search engine query recommendation. Neurocomputing 218(2016), 423–431.

Digital Library

[24]

Weiqing Wang, Hongzhi Yin, Zi Huang, Qinyong Wang, Xingzhong Du, and Quoc Viet Hung Nguyen. 2018. Streaming ranking based recommender systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 525–534.

Digital Library

[25]

Yichao Wang, Huifeng Guo, Ruiming Tang, Zhirong Liu, and Xiuqiang He. 2020. A Practical Incremental Method to Train Deep CTR Models. arXiv preprint arXiv:2009.02147(2020).

[26]

Junfeng Wen, Yanshuai Cao, and Ruitong Huang. 2018. Few-shot self reminder to overcome catastrophic forgetting. arXiv preprint arXiv:1812.00543(2018).

[27]

Yishi Xu, Yingxue Zhang, Wei Guo, Huifeng Guo, Ruiming Tang, and Mark Coates. 2020. GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2861–2868.

Digital Library

[28]

Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In International Conference on Machine Learning. PMLR, 3987–3995.

[29]

Yang Zhang, Fuli Feng, Chenxu Wang, Xiangnan He, Meng Wang, Yan Li, and Yongdong Zhang. 2020. How to retrain recommender system? A sequential meta-learning method. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1479–1488.

[30]

Yan Zhao, Shoujin Wang, Yan Wang, and Hongwei Liu. 2020. Stratified and time-aware sampling based adaptive ensemble learning for streaming recommendations. Applied Intelligence(2020), 1–21.

[31]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059–1068.

Digital Library

Cited By

Zhang KWang YLi XTang RZhang RAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)IncMSR: An Incremental Learning Approach for Multi-Scenario RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635828(939-948)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635828
Do JLauw H(2023)Continual Collaborative Filtering Through Gradient AlignmentProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3610648(1133-1138)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3610648
Yang CChen JYu QWu XMa KZhao ZFang ZChen WFan CHe JPeng CLin ZShao JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)An Incremental Update Framework for Online Recommenders with Data-Driven PriorProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615456(4894-4900)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615456
Show More Cited By

Recommendations

An Efficient Hybrid Music Recommender System Using an Incrementally Trainable Probabilistic Generative Model

This paper presents a hybrid music recommender system that ranks musical pieces while efficiently maintaining collaborative and content-based data, i.e., rating scores given by users and acoustic features of audio signals. This hybrid approach overcomes ...
Contrastive Meta Learning with Behavior Multiplicity for Recommendation
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

A well-informed recommendation framework could not only help users identify their interested items, but also benefit the revenue of various online platforms (e.g., e-commerce, social media). Traditional recommendation models usually assume that only a ...
Model-driven engineering with domain-specific meta-modelling languages

Domain-specific modelling languages are normally defined through general-purpose meta-modelling languages like the MOF. While this is satisfactory for many model-driven engineering (MDE) projects, several researchers have identified the need for domain-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems

September 2021

883 pages

ISBN:9781450384582

DOI:10.1145/3460231

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Alibaba-NTU Singapore Joint Research Institute

Conference

RecSys '21

Sponsor:

RecSys '21: Fifteenth ACM Conference on Recommender Systems

September 27 - October 1, 2021

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Upcoming Conference

RecSys '24

Sponsor:
sigchi

18th ACM Conference on Recommender Systems

October 14 - 18, 2024

Bari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
632
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)4

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang KWang YLi XTang RZhang RAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)IncMSR: An Incremental Learning Approach for Multi-Scenario RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635828(939-948)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635828
Do JLauw H(2023)Continual Collaborative Filtering Through Gradient AlignmentProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3610648(1133-1138)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3610648
Yang CChen JYu QWu XMa KZhao ZFang ZChen WFan CHe JPeng CLin ZShao JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)An Incremental Update Framework for Online Recommenders with Data-Driven PriorProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615456(4894-4900)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615456
Zhu JCai GHuang JDong ZTang RZhang WSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)ReLoop2: Building Self-Adaptive Recommendation Models via Responsive Error Compensation LoopProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599785(5728-5738)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599785
Ji YSun AZhang JLi C(2023)A Critical Study on Data Leakage in Recommender System Offline EvaluationACM Transactions on Information Systems10.1145/356993041:3(1-27)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3569930
Kolikipogu RDarak VYennapu RReddy SSureddi RKuchipudi R(2023)Agriculture Recommender System for Precision Farming using Machine Learning(ARS)2023 3rd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA)10.1109/ICIMIA60377.2023.10426510(921-927)Online publication date: 21-Dec-2023
https://doi.org/10.1109/ICIMIA60377.2023.10426510
Wang ZShen Y(2023)Incremental Learning for Multi-Interest Sequential Recommendation2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00087(1071-1083)Online publication date: May-2023
https://doi.org/10.1109/ICDE55515.2023.00087
Yoon JJang B(2023)Evolution of Deep Learning-Based Sequential Recommender Systems: From Current Trends to New PerspectivesIEEE Access10.1109/ACCESS.2023.328198111(54265-54279)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3281981
Guan RPang HGiunchiglia FLi XYang XFeng XAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Deployable and Continuable Meta-learning-Based Recommender System with Fast User-Incremental UpdatesProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531964(1423-1433)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531964
Lee SShin HChoi IKim J(2022)Analyzing the Impact of Components of Yelp.com on Recommender System Performance: Case of AustinIEEE Access10.1109/ACCESS.2022.322519010(128066-128076)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3225190
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents