research-article

Sinkhorn Collaborative Filtering

Authors:

Gao CongAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

Pages 582 - 592

https://doi.org/10.1145/3442381.3449869

Published: 03 June 2021 Publication History

Abstract

Recommender systems play a vital role in modern web services. In a typical recommender system, we are given a set of observed user-item interaction records and seek to uncover the hidden behavioral patterns of users from these historical interactions. By exploiting these hidden patterns, we aim to discover users’ personalized tastes and recommend them new items. Among various types of recommendation methods, the latent factor collaborative filtering models have dominated the field. In this paper, we develop a unified view for the existing latent factor models from a probabilistic perspective. The unified framework enables us to discern the underlying connections of different latent factor models and deepen our understandings of their advantages and limitations. In particular, we observe that the loss functions adopted by the existing models are oblivious to the geometry induced by the item-similarity. To address this, we propose a novel model—SinkhornCF—based on Sinkhorn divergence. To address the challenge of the expensive computational cost of Sinkhorn divergence, we also propose new techniques to enable the resulting model to be able to scale to large datasets. Its effectiveness is verified on two real-world recommendation datasets.

References

[1]

Martín Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. CoRR (2017).

[2]

Federico Bassetti, Antonella Bodini, and Eugenio Regazzini. 2006. On minimum Kantorovich distance estimators. Statistics & probability letters 76, 12 (2006), 1298–1302.

[3]

Léon Bottou, Frank E. Curtis, and Jorge Nocedal. 2018. Optimization Methods for Large-Scale Machine Learning. SIAM Rev. 60, 2 (2018), 223–311.

[4]

Jin Yao Chin, Kaiqi Zhao, Shafiq R. Joty, and Gao Cong. 2018. ANR: Aspect-based Neural Recommender. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, October 22-26, 2018. ACM, 147–156.

Digital Library

[5]

Colin Cooper, Sang Hyuk Lee, Tomasz Radzik, and Yiannis Siantos. 2014. Random Walks in Recommender Systems: Exact Computation and Simulations. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW ’14 Companion). Association for Computing Machinery, New York, NY, USA, 811–816. https://doi.org/10.1145/2567948.2579244

Digital Library

[6]

Marco Cuturi. 2013. Sinkhorn Distances: Lightspeed Computation of Optimal Transport. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger (Eds.). 2292–2300.

[7]

Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys ’19). Association for Computing Machinery, New York, NY, USA, 101–109. https://doi.org/10.1145/3298689.3347058

Digital Library

[8]

Gintare Karolina Dziugaite and Daniel M. Roy. 2015. Neural Network Matrix Factorization. CoRR abs/1511.06443(2015).

[9]

Aude Genevay, Gabriel Peyré, and Marco Cuturi. 2018. Learning Generative Models with Sinkhorn Divergences. In International Conference on Artificial Intelligence and Statistics, AISTATS 2018, 9-11 April 2018, Playa Blanca, Lanzarote, Canary Islands, Spain(Proceedings of Machine Learning Research, Vol. 84), Amos J. Storkey and Fernando Pérez-Cruz (Eds.). PMLR, 1608–1617.

[10]

Prem Gopalan, Jake M. Hofman, and David M. Blei. 2015. Scalable Recommendation with Hierarchical Poisson Factorization. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, UAI 2015, July 12-16, 2015, Amsterdam, The Netherlands, Marina Meila and Tom Heskes (Eds.). AUAI Press, 326–335. http://auai.org/uai2015/proceedings/papers/208.pdf

[11]

Prem K Gopalan, Laurent Charlin, and David Blei. 2014. Content-based recommendations with poisson factorization. In NIPS 2014. 3176–3184.

Digital Library

[12]

Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville. 2017. Improved Training of Wasserstein GANs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA. 5767–5777.

[13]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In Proceedings of the 26th International Conference on World Wide Web(WWW ’17). 173–182.

Digital Library

[14]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets(ICDM ’08). IEEE Computer Society, USA, 263–272. https://doi.org/10.1109/ICDM.2008.22

[15]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015.

[16]

Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings.

[17]

Mark Levy and Kris Jack. 2013. Efficient Top-N Recommendation by Linear Regression. (2013).

[18]

Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. https://doi.org/10.1145/3178876.3186150

Digital Library

[19]

G. Linden, B. Smith, and J. York. 2003. Amazon.com Recommendations: Item-to-Item Collaborative Filtering. IEEE Internet Computing 7, 1 (2003), 76–80.

Digital Library

[20]

James Robert Lloyd and Zoubin Ghahramani. 2015. Statistical Model Criticism using Kernel Two Sample Tests. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (Eds.). 829–837.

[21]

Benjamin Marlin. 2004. Collaborative filtering: A machine learning perspective. University of Toronto Toronto.

[22]

Yitong Meng, Xiao Yan, Weiwen Liu, Huanhuan Wu, and James Cheng. 2020. Wasserstein Collaborative Filtering for Item Cold-start Recommendation. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 318–322.

Digital Library

[23]

Grégoire Montavon, Klaus-Robert Müller, and Marco Cuturi. 2016. Wasserstein Training of Restricted Boltzmann Machines. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain. 3711–3719.

[24]

Jonas Mueller and Tommi S. Jaakkola. 2015. Principal Differences Analysis: Interpretable Characterization of Differences between Distributions. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. 1702–1710.

[25]

Xia Ning and George Karypis. 2011. SLIM: Sparse Linear Methods for Top-N Recommender Systems. In Proceedings of the 2011 IEEE 11th International Conference on Data Mining(ICDM ’11). IEEE Computer Society, USA, 497–506. https://doi.org/10.1109/ICDM.2011.134

Digital Library

[26]

Bibek Paudel, Fabian Christoffel, Chris Newell, and Abraham Bernstein. 2016. Updatable, Accurate, Diverse, and Scalable Recommendations for Interactive Applications. 7, 1, Article 1 (Dec. 2016), 34 pages. https://doi.org/10.1145/2955101

[27]

Gabriel Peyré and Marco Cuturi. 2019. Computational Optimal Transport. Found. Trends Mach. Learn. 11, 5-6 (2019), 355–607.

Digital Library

[28]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In ICML 2014, Beijing, China, 21-26 June 2014. 1278–1286.

[29]

Antoine Rolet, Marco Cuturi, and Gabriel Peyré. 2016. Fast Dictionary Learning with a Smoothed Wasserstein Loss. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016, Cadiz, Spain, May 9-11, 2016(JMLR Workshop and Conference Proceedings, Vol. 51), Arthur Gretton and Christian C. Robert (Eds.). JMLR.org, 630–638.

[30]

Ruslan Salakhutdinov and Andriy Mnih. 2007. Probabilistic Matrix Factorization. In Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3-6, 2007, John C. Platt, Daphne Koller, Yoram Singer, and Sam T. Roweis (Eds.). 1257–1264.

[31]

Tim Salimans, Han Zhang, Alec Radford, and Dimitris N. Metaxas. 2018. Improving GANs Using Optimal Transport. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.

[32]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-Based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th International Conference on World Wide Web(WWW ’01). 285–295.

Digital Library

[33]

Gábor Takács, István Pilászy, Bottyán Németh, and Domonkos Tikk. 2008. Matrix factorization and neighbor based algorithms for the netflix prize problem. In Proceedings of the 2008 ACM Conference on Recommender Systems, RecSys 2008, Lausanne, Switzerland, October 23-25, 2008, Pearl Pu, Derek G. Bridge, Bamshad Mobasher, and Francesco Ricci (Eds.). ACM, 267–274. https://doi.org/10.1145/1454008.1454049

Digital Library

[34]

Lucas Vinh Tran, Yi Tay, Shuai Zhang, Gao Cong, and Xiaoli Li. 2020. HyperML: A Boosting Metric Learning Approach in Hyperbolic Space for Recommender Systems. In WSDM ’20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020. ACM, 609–617. https://doi.org/10.1145/3336191.3371850

[35]

Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative Denoising Auto-Encoders for Top-N Recommender Systems. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, San Francisco, CA, USA, February 22-25, 2016, Paul N. Bennett, Vanja Josifovski, Jennifer Neville, and Filip Radlinski (Eds.). ACM, 153–162.

Digital Library

[36]

Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep Learning Based Recommender System: A Survey and New Perspectives. ACM Comput. Surv. 52, 1 (2019), 5:1–5:38.

Digital Library

Cited By

Lin ZYan QLiu WWang SWang MTan YYang C(2024)Automatic Hypergraph Generation for Enhancing Recommendation With Sparse OptimizationIEEE Transactions on Multimedia10.1109/TMM.2023.333808326(5680-5693)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3338083
Bai TWang XZhang ZSong WWu BNie J(2024)GPR-OPTInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10352561:1Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ipm.2023.103525
Li XMa CLi GXu PLiu CYuan YWang G(2023)Meta Auxiliary Learning for Top-K RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.322315535:10(10857-10870)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TKDE.2022.3223155

Recommendations

Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

User-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
Modelling socially-influenced conditional preferences over feature values in recommender systems based on factorised collaborative filtering

Novel method for modelling conditional preferences over feature values is proposed.The social influence on the components of preferences is taken into consideration.Extensive experiments and statistical analysis carried out to evaluate our model.Our ...
Typicality-Based Collaborative Filtering Recommendation

Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
270
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin ZYan QLiu WWang SWang MTan YYang C(2024)Automatic Hypergraph Generation for Enhancing Recommendation With Sparse OptimizationIEEE Transactions on Multimedia10.1109/TMM.2023.333808326(5680-5693)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3338083
Bai TWang XZhang ZSong WWu BNie J(2024)GPR-OPTInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10352561:1Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ipm.2023.103525
Li XMa CLi GXu PLiu CYuan YWang G(2023)Meta Auxiliary Learning for Top-K RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.322315535:10(10857-10870)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TKDE.2022.3223155

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents