research-article

Factorization Machines with libFM

Author:

Steffen RendleAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 3, Issue 3

Article No.: 57, Pages 1 - 22

https://doi.org/10.1145/2168752.2168771

Published: 01 May 2012 Publication History

Abstract

Factorization approaches provide high accuracy in several important prediction problems, for example, recommender systems. However, applying factorization approaches to a new prediction problem is a nontrivial task and requires a lot of expert knowledge. Typically, a new model is developed, a learning algorithm is derived, and the approach has to be implemented.

Factorization machines (FM) are a generic approach since they can mimic most factorization models just by feature engineering. This way, factorization machines combine the generality of feature engineering with the superiority of factorization models in estimating interactions between categorical variables of large domain. libFM is a software implementation for factorization machines that features stochastic gradient descent (SGD) and alternating least-squares (ALS) optimization, as well as Bayesian inference using Markov Chain Monto Carlo (MCMC). This article summarizes the recent research on factorization machines both in terms of modeling and learning, provides extensions for the ALS and MCMC algorithms, and describes the software tool libFM.

References

[1]

Adomavicius, G., Sankaranarayanan, R., Sen, S., and Tuzhilin, A. 2005. Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Info. Syst. 23, 1, 103--145.

Digital Library

[2]

Agarwal, D. and Chen, B.-C. 2009. Regression-based latent factor models. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’09). ACM, New York, NY, 19--28.

Digital Library

[3]

Baltrunas, L. and Ricci, F. 2009. Context-based splitting of item ratings in collaborative filtering. In Proceedings of the third ACM Conference on Recommender Systems (RecSys’09). ACM, New York, NY, 245--248.

Digital Library

[4]

Chang, C.-C. and Lin, C.-J. 2011. Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1--27:27.

Digital Library

[5]

Chen, T., Zheng, Z., Lu, Q., Zhang, W., and Yu, Y. 2011. Feature-based matrix factorization. Tech. rep. APEX-TR-2011-07-11, Apex Data & Knowledge Management Lab, Shanghai Jiao Tong University.

[6]

Freudenthaler, C., Schmidt-Thieme, L., and Rendle, S. 2011. Bayesian factorization machines. In Proceedings of the NIPS Workshop on Sparse Representation and Low-rank Approximation.

[7]

Gantner, Z., Drumond, L., Freudenthaler, C., Rendle, S., and Lars, S.-T. 2010. Learning attribute-to-feature mappings for cold-start recommendations. In Proceedings of the IEEE International Conference on Data Mining (ICDM’10). IEEE Computer Society, Los Alamintos, CA, 176--185.

Digital Library

[8]

Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. 2003. Bayesian Data Analysis 2nd Ed. Chapman and Hall/CRC.

[9]

Gunawardana, A. and Shani, G. 2009. A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10, 2935--2962.

Digital Library

[10]

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. 2009. The weka data mining software: An update. SIGKDD Explor. Newsl. 11, 10--18.

Digital Library

[11]

Harshman, R. A. 1970. Foundations of the parafac procedure: Models and conditions for an ‘exploratory’ multimodal factor analysis. UCLA Working Papers in Phonetics, 1--84.

[12]

Jahrer, M., Töscher, A., and Legenstein, R. 2010. Combining predictions for accurate recommender systems. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10). ACM, New York, NY, 693--702.

Digital Library

[13]

Joachims, T. 1999. Making Large-Scale Support Vector Machine Learning Practical. MIT Press, Cambridge, MA, 169--184.

Digital Library

[14]

Karatzoglou, A., Amatriain, X., Baltrunas, L., and Oliver, N. 2010. Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). ACM, New York, NY, 79--86.

Digital Library

[15]

Koren, Y. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’08). ACM, New York, NY, 426--434.

Digital Library

[16]

Koren, Y. 2009a. The bellkor solution to the Netflix grand prize.

[17]

Koren, Y. 2009b. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’09). ACM, New York, NY, 447--456.

Digital Library

[18]

Koren, Y. 2010. Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Trans. Knowl. Discov. Data 4, 1:1--1:24.

Digital Library

[19]

Lim, Y. J. and Teh, Y. W. 2007. Variational Bayesian approach to movie rating prediction. In Proceedings of the KDD Cup and Workshop.

[20]

Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. Inter. Comput. IEEE 7, 1, 76--80.

Digital Library

[21]

Lipczak, M., Hu, Y., Kollet, Y., and Milios, E. 2009. Tag sources for recommendation in collaborative tagging systems. In Proceedings of the ECML-PKDD Discovery Challenge Workshop.

[22]

Lipczak, M. and Milios, E. 2011. Efficient tag recommendation for real-life data. ACM Trans. Intell. Syst. Technol. 3, 1, 2:1--2:21.

Digital Library

[23]

Liu, N. N. and Yang, Q. 2008. Eigenrank: A ranking-oriented approach to collaborative filtering. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’08). ACM, New York, NY, 83--90.

Digital Library

[24]

Ma, H., King, I., and Lyu, M. R. 2011. Learning to recommend with explicit and implicit social relations. ACM Trans. Intell. Syst. Technol. Article 29.

Digital Library

[25]

Marinho, L. B., Preisach, C., and Schmidt-Thieme, L. 2009. Relational classification for personalized tag recommendation. In Proceedings of the ECML-PKDD Discovery Challenge Workshop.

[26]

Paterek, A. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of the KDD Cup Workshop 13th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD’07). 39--42.

[27]

Pilászy, I., Zibriczky, D., and Tikk, D. 2010. Fast als-based matrix factorization for explicit and implicit feedback datasets. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). ACM, New York, NY, 71--78.

Digital Library

[28]

Rendle, S. 2010. Factorization machines. In Proceedings of the 10th IEEE International Conference on Data Mining. IEEE Computer Society.

Digital Library

[29]

Rendle, S. 2012. Learning recommender systems with adaptive regularization. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining (WSDM’12). ACM, New York, NY, 133--142.

Digital Library

[30]

Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI’09).

Digital Library

[31]

Rendle, S., Freudenthaler, C., and Schmidt-Thieme, L. 2010. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM, New York, NY, 811--820.

Digital Library

[32]

Rendle, S., Gantner, Z., Freudenthaler, C., and Schmidt-Thieme, L. 2011. Fast context-aware recommendations with factorization machines. In Proceedings of the 34th ACM SIGIR Conference on Reasearch and Development in Information Retrieval.

Digital Library

[33]

Rendle, S. and Schmidt-Thieme, L. 2010. Pairwise interaction tensor factorization for personalized tag recommendation. In Proceedings of the third ACM International Conference on Web Search and Data Mining (WSDM’10). ACM, New York, NY, 81--90.

Digital Library

[34]

Robert, C. P. 1995. Simulation of truncated normal variables. Stat. Comput. 5, 121--125.

[35]

Salakhutdinov, R. and Mnih, A. 2008a. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In Proceedings of the 25th International Conference on Machine Learning.

Digital Library

[36]

Salakhutdinov, R. and Mnih, A. 2008b. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems 20.

[37]

Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. ACM Press, New York, NY, 285--295.

Digital Library

[38]

Srebro, N. and Jaakkola, T. 2003. Weighted low rank approximation. In Proceedings of the 20th International Conference on Machine Learning (ICML’03).

[39]

Srebro, N., Rennie, J. D. M., and Jaakola, T. S. 2005. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 17, MIT 1329--1336.

[40]

Stern, D. H., Herbrich, R., and Graepel, T. 2009. Matchbox: Large-scale online Bayesian recommendations. In Proceedings of the 18th International Conference on World Wide Web (WWW’09). ACM, New York, NY, 111--120.

Digital Library

[41]

Takács, G., Pilászy, I., Németh, B., and Tikk, D. 2009. Scalable collaborative filtering approaches for large recommender systems. J. Mach. Learn. Res. 10, 623--656.

Digital Library

[42]

Tucker, L. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279--311.

[43]

Xiong, L., Chen, X., Huang, T.-K., Schneider, J., and Carbonell, J. G. 2010. Temporal collaborative filtering with Bayesian probabilistic tensor factorization. In Proceedings of the SIAM International Conference on Data Mining (SIAM). 211--222.

[44]

Zhang, N., Zhang, Y., and Tang, J. 2009. A tag recommendation system based on contents. In Proceedings of the ECML-PKDD Discovery Challenge Workshop.

[45]

Zheng, Y. and Xie, X. 2011. Learning travel recommendations from user-generated gps traces. ACM Trans. Intell. Syst. Technol. 2, 2:1--2:29.

Digital Library

Cited By

Zheng CSong Y(2025)Personalized multi-head self-attention network for news recommendationNeural Networks10.1016/j.neunet.2024.106824181(106824)Online publication date: Jan-2025
https://doi.org/10.1016/j.neunet.2024.106824
Jiang MLi MCao WYang MZhou L(2025)Multi-task convolutional deep neural network for recommendation based on knowledge graphsNeurocomputing10.1016/j.neucom.2024.129136619(129136)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.129136
Chen ZXie DWang XYou DShen L(2025)High-order complementary cloud application programming interface recommendation with logical reasoning for incremental developmentEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109698140(109698)Online publication date: Jan-2025
https://doi.org/10.1016/j.engappai.2024.109698
Show More Cited By

Index Terms

Factorization Machines with libFM

Recommendations

Co-manifold Matrix Factorization
ICCPR '20: Proceedings of the 2020 9th International Conference on Computing and Pattern Recognition

Matrix factorization plays a fundamental role in collaborative filtering. In collaborative filtering setting, the rating matrix R is very sparse. Thus, infinite number of matrices can fit the observed entries in the rating matrix. Without additional ...
Factorization Machines
ICDM '10: Proceedings of the 2010 IEEE International Conference on Data Mining

In this paper, we introduce Factorization Machines (FM) which are a new model class that combines the advantages of Support Vector Machines (SVM) with factorization models. Like SVMs, FMs are a general predictor working with any real valued feature ...
Separating-Plane Factorization Models: Scalable Recommendation from One-Class Implicit Feedback
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

We study the video recommendation problem based on a large amount of user viewing logs instead of explicit ratings. As viewing records are implicitly suggest user preferences, existing matrix factorization methods fail to generate discriminative ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 3, Issue 3

May 2012

384 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/2168752

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 2012

Accepted: 01 February 2012

Revised: 01 January 2012

Received: 01 January 2012

Published in TIST Volume 3, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Survey
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

958
Total Citations
View Citations
12,114
Total Downloads

Downloads (Last 12 months)370
Downloads (Last 6 weeks)62

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zheng CSong Y(2025)Personalized multi-head self-attention network for news recommendationNeural Networks10.1016/j.neunet.2024.106824181(106824)Online publication date: Jan-2025
https://doi.org/10.1016/j.neunet.2024.106824
Jiang MLi MCao WYang MZhou L(2025)Multi-task convolutional deep neural network for recommendation based on knowledge graphsNeurocomputing10.1016/j.neucom.2024.129136619(129136)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.129136
Chen ZXie DWang XYou DShen L(2025)High-order complementary cloud application programming interface recommendation with logical reasoning for incremental developmentEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109698140(109698)Online publication date: Jan-2025
https://doi.org/10.1016/j.engappai.2024.109698
Arockia Venice JArivazhagan DSuman NShanthi HSwadhi R(2024)Recommendation Systems and Content PersonalizationAI for Large Scale Communication Networks10.4018/979-8-3693-6552-6.ch015(323-348)Online publication date: 25-Oct-2024
https://doi.org/10.4018/979-8-3693-6552-6.ch015
Gao QSugawara MNation PKobayashi TOhnishi YTezuka HYamamoto N(2024)A Quantum-Classical Method Applied to Material Design: Photochromic Materials Optimization for Photopharmacology ApplicationsIntelligent Computing10.34133/icomputing.01083Online publication date: 20-Dec-2024
https://doi.org/10.34133/icomputing.0108
Kim MLee SKim J(2024)Explainable Neural Tensor Factorization for Commercial Alley Revenues PredictionElectronics10.3390/electronics1316327913:16(3279)Online publication date: 19-Aug-2024
https://doi.org/10.3390/electronics13163279
Jiang YGao YSun YWang SYan C(2024)Multi-Channel Hypergraph Collaborative Filtering with Attribute InferenceElectronics10.3390/electronics1305090313:5(903)Online publication date: 27-Feb-2024
https://doi.org/10.3390/electronics13050903
Yin LZhang HHe R(2024)Game Difficulty Prediction Based on Facial Cues and Game PerformanceApplied Sciences10.3390/app1419877814:19(8778)Online publication date: 28-Sep-2024
https://doi.org/10.3390/app14198778
Zhang DZhu ZWang ZWang JXiao LChen YZhao D(2024)Personalized News Recommendation Method with Double-Layer Residual Connections and Double Multi-Head Self-Attention MechanismsApplied Sciences10.3390/app1413566714:13(5667)Online publication date: 28-Jun-2024
https://doi.org/10.3390/app14135667
Tahir Kidwai UAkhtar NNadeem MAlroobaea R(2024)Mitigating filter bubbles: Diverse and explainable recommender systemsJournal of Intelligent & Fuzzy Systems10.3233/JIFS-219416(1-14)Online publication date: 26-Apr-2024
https://doi.org/10.3233/JIFS-219416
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents