research-article

Public Access

AdaError: An Adaptive Learning Rate Method for Matrix Approximation-based Collaborative Filtering

Authors:

Stephen M. ChuAuthors Info & Claims

WWW '18: Proceedings of the 2018 World Wide Web Conference

Pages 741 - 751

https://doi.org/10.1145/3178876.3186155

Published: 23 April 2018 Publication History

All formats PDF

Abstract

Gradient-based learning methods such as stochastic gradient descent are widely used in matrix approximation-based collaborative filtering algorithms to train recommendation models based on observed user-item ratings. One major difficulty in existing gradient-based learning methods is determining proper learning rates, since model convergence would be inaccurate or very slow if the learning rate is too large or too small, respectively. This paper proposes AdaError, an adaptive learning rate method for matrix approximation-based collaborative filtering. AdaError eliminates the need of manually tuning the learning rates by adaptively adjusting the learning rates based on the noisiness level of user-item ratings, using smaller learning rates for noisy ratings so as to reduce their impact on the learned models. Our theoretical and empirical analysis shows that AdaError can improve the generalization performance of the learned models. Experimental studies on the MovieLens and Netflix datasets also demonstrate that AdaError outperforms state-of-the-art adaptive learning rate methods in matrix approximation-based collaborative filtering. Furthermore, by applying AdaError to the standard matrix approximation method, we can achieve statistically significant improvements over state-of-the-art collaborative filtering methods in both rating prediction accuracy and top-N recommendation accuracy.

References

[1]

Xavier Amatriain, Josep M. Pujol, and Nuria Oliver. 2009. I Like It... I Like It Not: Evaluating User Ratings Noise in Recommender Systems. In Proceedings of the 17th International Conference on User Modeling, Adaptation, and Personalization (UMAP '09). Springer, 247--258.

Digital Library

[2]

Xavier Amatriain, Josep M. Pujol, Nava Tintarev, and Nuria Oliver. 2009. Rate It Again: Increasing Recommendation Accuracy by User Re-rating. In Proceedings of the Third ACM Conference on Recommender Systems (RecSys '09). ACM, 173--180.

Digital Library

[3]

Alex Beutel, Amr Ahmed, and Alexander J. Smola. 2015. ACCAMS: Additive Co-Clustering to Approximate Matrices Succinctly. In Proceedings of the 24th International Conference on World Wide Web (WWW '15). 119--129.

Digital Library

[4]

Alex Beutel, Ed H. Chi, Zhiyuan Cheng, Hubert Pham, and John Anderson. 2017. Beyond Globally Optimal: Focused Learning for Improved Recommendations. In Proceedings of the 26th International Conference on World Wide Web (WWW '17). 203--212.

Digital Library

[5]

Daniel Billsus and Michael J Pazzani. 1998. Learning Collaborative Information Filters. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98), Vol. 98. 46--54.

Digital Library

[6]

Olivier Bousquet and Andr´e Elisseeff. 2001. Algorithmic Stability and Generalization Performance. In Advances in Neural Information Processing Systems. 196--202.

Digital Library

[7]

Emmanuel J. Cand

[8]

es and Yaniv Plan. 2010. Matrix Completion With Noise. Proc. IEEE 98, 6 (2010), 925--936.

[9]

Chao Chen, Dongsheng Li, Yingying Zhao, Qin Lv, and Li Shang. 2015. WEMAREC: Accurate and Scalable Recommendation through Weighted and Ensemble Matrix Approximation. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 303--312.

Digital Library

[10]

Peizhe Cheng, Shuaiqiang Wang, Jun Ma, Jiankai Sun, and Hui Xiong. 2017. Learning to Recommend Accurate and Diverse Items. In Proceedings of the 26th International Conference on World Wide Web (WWW '17). 183--192.

Digital Library

[11]

Dan Cosley, Shyong K. Lam, Istvan Albert, Joseph A. Konstan, and John Riedl. 2003. Is Seeing Believing?: How Recommender System Interfaces Affect Users? Opinions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, 585--592.

Digital Library

[12]

Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223--1231.

Digital Library

[13]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, Jul (2011), 2121--2159.

Digital Library

[14]

Moritz Hardt, Benjamin Recht, and Yoram Singer. 2016. Train Faster, Generalize Better: Stability of Stochastic Gradient Descent. In Proceedings of the 33rd International Conference on International Conference on Machine Learning (ICML'16). JMLR.org, 1225--1234.

Digital Library

[15]

Elad Hazan and Satyen Kale. 2014. Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimization. Journal of Machine Learning Research 15, 1 (2014), 2489--2512.

Digital Library

[16]

Liang Hu, Jian Cao, Guandong Xu, Longbing Cao, Zhiping Gu, and Can Zhu. 2013. Personalized Recommendation via Crossdomain Triadic Factorization. In Proceedings of the 22Nd International Conference on World Wide Web (WWW '13). ACM, 595--606.

Digital Library

[17]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. In Proceedings of the Eighth IEEE International Conference on Data Mining (ICDM '08). 263--272.

Digital Library

[18]

Robert A Jacobs. 1988. Increased rates of convergence through learning rate adaptation. Neural networks 1, 4 (1988), 295--307.

[19]

Nicolas Jones, Armelle Brun, and Anne Boyer. 2011. Comparisons Instead of Ratings: Towards More Stable Preferences. In Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intel ligence and Intel ligent Agent Technology (WI-IAT '11). IEEE, 451--456.

Digital Library

[20]

Santosh Kabbur, Xia Ning, and George Karypis. 2013. FISM: Factored Item Similarity Models for top-N Recommender Systems. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '13). ACM, 659--667.

Digital Library

[21]

Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[22]

Yehuda Koren. 2008. Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '08). ACM, 426--434.

Digital Library

[23]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30--37.

Digital Library

[24]

Balaji Lakshminarayanan, Guillaume Bouchard, and Cedric Archambeau. 2011. Robust Bayesian matrix factorisation. In Proceedings of the International Conference on Artificial Intel ligence and Statistics (AISTATS). 425--433.

[25]

Joonseok Lee, Seungyeon Kim, Guy Lebanon, and Yoram Singer. 2013. Local low-rank matrix approximation. In Proceedings of The 30th International Conference on Machine Learning (ICML '13). 82--90.

Digital Library

[26]

Dongsheng Li, Chao Chen, Qin Lv, Li Shang, Stephen M. Chu, and Hongyuan Zha. 2017. ERMMA: Expected Risk Minimization for Matrix Approximation-based Recommender Systems. In Proceedings of the Thirty-First AAAI Conference on Artificial Intel ligence (AAAI '17). 1403--1409.

[27]

Dongsheng Li, Chao Chen, Qin Lv, Junchi Yan, Li Shang, and Stephen M. Chu. 2016. Low-rank matrix approximation with stability. In Proceedings of The 33rd International Conference on Machine Learning (ICML '16). 295--303.

Digital Library

[28]

Lester W Mackey, Michael I Jordan, and Ameet Talwalkar. 2011. Divide-and-conquer matrix factorization. In Advances in Neural Information Processing Systems. 1134--1142.

Digital Library

[29]

Bhaskar Mehta, Thomas Hofmann, and Wolfgang Nejdl. 2007. Robust Collaborative Filtering. In Proceedings of the 2007 ACM Conference on Recommender Systems (RecSys '07). ACM, 49-- 56.

Digital Library

[30]

Andrew Y. Ng. 2004. Feature Selection, L1 vs. L2 Regularization, and Rotational Invariance. In Proceedings of the Twenty-first International Conference on Machine Learning (ICML '04). 78--85.

Digital Library

[31]

Tien T. Nguyen, Daniel Kluver, Ting-Yu Wang, Pik-Mai Hui, Michael D. Ekstrand, Martijn C. Willemsen, and John Riedl. 2013. Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys '13). ACM, 149-- 156.

Digital Library

[32]

Xia Ning and George Karypis. 2011. SLIM: Sparse Linear Methods for Top-N Recommender Systems. In Proceedings of the 2011 IEEE 11th International Conference on Data Mining (ICDM '11). 497--506.

Digital Library

[33]

Michael O'Mahony, Neil Hurley, Nicholas Kushmerick, and Gu´enol´e Silvestre. 2004. Collaborative Recommendation: A Robustness Analysis. ACM Trans. Internet Technol. 4, 4 (2004), 344--377.

Digital Library

[34]

Michael P. O'Mahony, Neil J. Hurley, and Gu´enol´e C.M. Silvestre. 2006. Detecting Noise in Recommender System Databases. In Proceedings of the 11th International Conference on Intel ligent User Interfaces (IUI '06). ACM, 109--115.

Digital Library

[35]

Arkadiusz Paterek. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of KDD cup and workshop, Vol. 2007. 5--8.

[36]

Steffen Rendle and Christoph Freudenthaler. 2014. Improving Pairwise Learning for Item Recommendation from Implicit Feedback. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM '14). 273--282.

Digital Library

[37]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intel ligence. 452--461.

Digital Library

[38]

Paul Resnick and Rahul Sami. 2007. The Influence Limiter: Provably Manipulation-resistant Recommender Systems. In Proceedings of the 2007 ACM Conference on Recommender Systems (RecSys '07). ACM, 25--32.

Digital Library

[39]

Ruslan Salakhutdinov and Andriy Mnih. 2008. Bayesian Probabilistic Matrix Factorization Using Markov Chain Monte Carlo. In Proceedings of the 25th International Conference on Machine Learning (ICML '08). ACM, 880--887.

Digital Library

[40]

Ruslan Salakhutdinov and Andriy Mnih. 2008. Probabilistic matrix factorization. In Advances in neural information processing systems. 1257--1264.

Digital Library

[41]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2000. Application of Dimensionality Reduction in Recommender System - A Case Study. In ACM WebKDD 2000 Workshop. ACM SIGKDD.

[42]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2002. Incremental Singular Value Decomposition Algorithms for Highly Scalable Recommender Systems. In Proceedings of the 5th International Conference in Computers and Information Technology.

[43]

Madeleine Udell, Corinne Horn, Reza Zadeh, and Stephen Boyd. 2016. Generalized Low Rank Models. Foundations and Trends in Machine Learning 9, 1 (2016), 1--118.

Digital Library

[44]

Linli Xu, Zaiyi Chen, Qi Zhou, Enhong Chen, Nicholas Jing Yuan, and Xing Xie. 2016. Aligned Matrix Completion: Integrating Consistency and Independency in Multiple Domains. In 2016 IEEE 16th International Conference on Data Mining (ICDM). 529--538.

[45]

Ting Yuan, Jian Cheng, Xi Zhang, Shuang Qiu, and Hanqing Lu. 2014. Recommendation by Mining Multiple User Behaviors with Group Sparsity. In Proceedings of the 28th AAAI Conference on Artificial Intel ligence (AAAI '14). 222--228.

Digital Library

[46]

Matthew D Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012).

[47]

Yongfeng Zhang, Min Zhang, Yiqun Liu, Shaoping Ma, and Shi Feng. 2013. Localized Matrix Factorization for Recommendation Based on Matrix Block Diagonal Forms. In Proceedings of the 22Nd International Conference on World Wide Web (WWW '13). ACM, 1511--1520.

Digital Library

Cited By

Elahi FFazlali MMalazi HElahi M(2024)Parallel Fractional Stochastic Gradient Descent With Adaptive Learning for Recommender SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.318521235:3(470-483)Online publication date: Mar-2024
https://doi.org/10.1109/TPDS.2022.3185212
Sha XSun ZZhang JOng Y(2024)Who Wants to Shop With You: Joint Product–Participant Recommendation for Group-Buying ServiceIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.319000335:2(2353-2363)Online publication date: Feb-2024
https://doi.org/10.1109/TNNLS.2022.3190003
Moustakas TTziouvaras AKolomvatsos K(2024)Data and resource aware incremental ML training in support of pervasive applicationsComputing10.1007/s00607-024-01338-2106:11(3727-3753)Online publication date: 16-Aug-2024
https://doi.org/10.1007/s00607-024-01338-2
Show More Cited By

Index Terms

AdaError: An Adaptive Learning Rate Method for Matrix Approximation-based Collaborative Filtering
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
  2. World Wide Web
    1. Web searching and information discovery
      1. Collaborative filtering

Recommendations

A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data Mining

Recommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
User preference representation based on psychometric models
ADC '11: Proceedings of the Twenty-Second Australasian Database Conference - Volume 115

Neighbourhood-based collaborative filtering is one of the most popular recommendation techniques, and has been applied successfully in various fields. User ratings are often used by neighbourhood-based collaborative filtering to compute the similarity ...
Improving Accuracy of Recommender System by Item Clustering

Recommender System (RS) predicts user's ratings towards items, and then recommends highly-predicted items to user. In recent years, RS has been playing more and more important role in the agent research field. There have been a great deal of researches ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '18: Proceedings of the 2018 World Wide Web Conference

April 2018

2000 pages

ISBN:9781450356398

General Chairs:
Pierre-Antoine Champin
Universitè Claude Bernard Lyon 1, France
,
Fabien Gandon
Inria, Université Côte d'Azur, CNRS, I3S, France
,
Lionel Médini
Université Claude Bernard Lyon 1, France
,
Program Chairs:
Mounia Lalmas
Spotify, UK
,
Panagiotis G. Ipeirotis
New York University, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

IW3C2: International World Wide Web Conference Committee

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

International World Wide Web Conferences Steering Committee

Republic and Canton of Geneva, Switzerland

Publication History

Published: 23 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
National Science Foundation of USA

Conference

WWW '18

Sponsor:

IW3C2

WWW '18: The Web Conference 2018

April 23 - 27, 2018

Lyon, France

Acceptance Rates

WWW '18 Paper Acceptance Rate 170 of 1,155 submissions, 15%;

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,474
Total Downloads

Downloads (Last 12 months)227
Downloads (Last 6 weeks)47

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Elahi FFazlali MMalazi HElahi M(2024)Parallel Fractional Stochastic Gradient Descent With Adaptive Learning for Recommender SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.318521235:3(470-483)Online publication date: Mar-2024
https://doi.org/10.1109/TPDS.2022.3185212
Sha XSun ZZhang JOng Y(2024)Who Wants to Shop With You: Joint Product–Participant Recommendation for Group-Buying ServiceIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.319000335:2(2353-2363)Online publication date: Feb-2024
https://doi.org/10.1109/TNNLS.2022.3190003
Moustakas TTziouvaras AKolomvatsos K(2024)Data and resource aware incremental ML training in support of pervasive applicationsComputing10.1007/s00607-024-01338-2106:11(3727-3753)Online publication date: 16-Aug-2024
https://doi.org/10.1007/s00607-024-01338-2
Chen JWang RWu DLuo X(2023)A Differential Evolution-Enhanced Position-Transitional Approach to Latent Factor AnalysisIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2022.31866737:2(389-401)Online publication date: Apr-2023
https://doi.org/10.1109/TETCI.2022.3186673
Li DLian JZhang LRen KLu TWu TXie XLi DLian JZhang LRen KLu TWu TXie X(2023)Overview of Recommender SystemsRecommender Systems10.1007/978-981-99-8964-5_1(1-30)Online publication date: 24-Nov-2023
https://doi.org/10.1007/978-981-99-8964-5_1
Luo XYuan YChen SZeng NWang Z(2022)Position-Transitional Particle Swarm Optimization-Incorporated Latent Factor AnalysisIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.303332434:8(3958-3970)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TKDE.2020.3033324
Chen JYi XHu YLiu YZhang R(2022)Accurate Latent Factor Analysis via Dynamic-Neighbor-cooperated Hierarchical Particle Swarm Optimizers2022 IEEE International Conference on Networking, Sensing and Control (ICNSC)10.1109/ICNSC55942.2022.10004108(1-6)Online publication date: 15-Dec-2022
https://doi.org/10.1109/ICNSC55942.2022.10004108
Chen JYi XHu YLiu YZhang R(2022)Accurate Latent Factor Analysis via Dynamic-Neighbor-cooperated Hierarchical Particle Swarm Optimizers2022 IEEE International Conference on Networking, Sensing and Control (ICNSC)10.1109/ICNSC55942.2022.10004091(1-6)Online publication date: 15-Dec-2022
https://doi.org/10.1109/ICNSC55942.2022.10004091
Yuan YLuo XYuan YLuo X(2022)Learning Rate-Free Latent Factor Analysis via PSOLatent Factor Analysis for High-dimensional and Sparse Matrices10.1007/978-981-19-6703-0_2(11-27)Online publication date: 16-Nov-2022
https://doi.org/10.1007/978-981-19-6703-0_2
Almutairi FSidiropoulos NYang BDemartini GZuccon GCulpepper JHuang ZTong H(2021)XPL-CFProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482221(2847-2851)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482221
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents