Uncertainty-adjusted Inductive Matrix Completion with Graph Neural Networks
Pages 1169 - 1174
Abstract
We propose a robust recommender systems model which performs matrix completion and a ratings-wise uncertainty estimation jointly. Whilst the prediction module is purely based on an implicit low-rank assumption imposed via nuclear norm regularization, our loss function is augmented by an uncertainty estimation module which learns an anomaly score for each individual rating via a Graph Neural Network: data points deemed more anomalous by the GNN are downregulated in the loss function used to train the low-rank module. The whole model is trained in an end-to-end fashion, allowing the anomaly detection module to tap on the supervised information available in the form of ratings. Thus, our model’s predictors enjoy the favourable generalization properties that come with being chosen from small function space (i.e., low-rank matrices), whilst exhibiting the robustness to outliers and flexibility that comes with deep learning methods. Furthermore, the anomaly scores themselves contain valuable qualitative information. Experiments on various real-life datasets demonstrate that our model outperforms standard matrix completion and other baselines, confirming the usefulness of the anomaly detection module.
References
[1]
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U Rajendra Acharya, 2021. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion 76 (2021), 243–297.
[2]
Rodrigo Alves, Antoine Ledent, and Marius Kloft. 2023. Uncertainty-Adjusted Recommendation via Matrix Factorization With Weighted Losses. IEEE Transactions on Neural Networks and Learning Systems (2023).
[3]
Martin Anthony, Peter L Bartlett, Peter L Bartlett, 1999. Neural network learning: Theoretical foundations. Vol. 9. cambridge university press Cambridge.
[4]
Albert-Laszlo Barabasi. 2005. The origin of bursts and heavy tails in human dynamics. Nature 435, 7039 (2005), 207–211.
[5]
Emmanuel J Candes and Yaniv Plan. 2010. Matrix completion with noise. Proc. IEEE 98, 6 (2010), 925–936.
[6]
Emmanuel J. Candès and Benjamin Recht. 2009. Exact Matrix Completion via Convex Optimization. Foundations of Computational Mathematics 9, 6 (2009), 717.
[7]
Emmanuel J. Candès and Terence Tao. 2010. The Power of Convex Relaxation: Near-Optimal Matrix Completion. IEEE Transactions on Information Theory 56, 5 (May 2010), 2053–2080.
[8]
Supriyo Chakraborty, Richard Tomsett, Ramya Raghavendra, Daniel Harborne, Moustafa Alzantot, Federico Cerutti, Mani Srivastava, Alun Preece, Simon Julier, Raghuveer M. Rao, Troy D. Kelley, Dave Braines, Murat Sensoy, Christopher J. Willis, and Prudhvi Gurram. 2017. Interpretability of deep learning models: A survey of results. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation. 1–6. https://doi.org/10.1109/UIC-ATC.2017.8397411
[9]
Alexander L Chistov and D Yu Grigor’Ev. 1984. Complexity of quantifier elimination in the theory of algebraically closed fields. In Mathematical Foundations of Computer Science 1984: Proceedings, 11th Symposium Praha, Czechoslovakia September 3–7, 1984 11. Springer, 17–31.
[10]
Dan Cosley, Shyong K Lam, Istvan Albert, Joseph A Konstan, and John Riedl. 2003. Is seeing believing? How recommender system interfaces affect users’ opinions. In Proceedings of the SIGCHI conference on Human factors in computing systems. 585–592.
[11]
Gintare Karolina Dziugaite and Daniel M Roy. 2015. Neural network matrix factorization. arXiv preprint arXiv:1511.06443 (2015).
[12]
Maryam Fazel. 2002. Matrix rank minimization with applications. Ph. D. Dissertation. PhD thesis, Stanford University.
[13]
Rina Foygel, Ohad Shamir, Nati Srebro, and Russ R Salakhutdinov. 2011. Learning with the weighted trace-norm under arbitrary sampling distributions. In Advances in Neural Information Processing Systems 24, J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2133–2141. http://papers.nips.cc/paper/4303-learning-with-the-weighted-trace-norm-under-arbitrary-sampling-distributions.pdf
[14]
Chen Gao, Yu Zheng, Nian Li, Yinfeng Li, Yingrong Qin, Jinghua Piao, Yuhan Quan, Jianxin Chang, Depeng Jin, Xiangnan He, 2023. A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Transactions on Recommender Systems 1, 1 (2023), 1–51.
[15]
Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 80–89.
[16]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In International Conference on Artificial Intelligence and Statistics.
[17]
F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (dec 2015), 19 pages. https://doi.org/10.1145/2827872
[18]
Trevor Hastie, Rahul Mazumder, Jason D. Lee, and Reza Zadeh. 2015. Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares. Journal of Machine Learning Research 16, 104 (2015), 3367–3402. http://jmlr.org/papers/v16/hastie15a.html
[19]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 639–648.
[20]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173–182.
[21]
Anthony Jameson, Martijn C Willemsen, Alexander Felfernig, Marco de Gemmis, Pasquale Lops, Giovanni Semeraro, and Li Chen. 2015. Human decision making and recommender systems. In Recommender systems handbook. Springer, 611–648.
[22]
Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision?Advances in neural information processing systems 30 (2017).
[23]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014).
[24]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
[25]
Quoc V Le, Alex J Smola, and Stéphane Canu. 2005. Heteroscedastic Gaussian process regression. In Proceedings of the 22nd international conference on Machine learning. 489–496.
[26]
Antoine Ledent, Rodrigo Alves, Yunwen Lei, and Marius Kloft. 2021. Fine-grained Generalization Analysis of Inductive Matrix Completion. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.). Vol. 34. Curran Associates, Inc., 25540–25552. https://proceedings.neurips.cc/paper/2021/file/d6428eecbe0f7dff83fc607c5044b2b9-Paper.pdf
[27]
George Lekakos and Petros Caravelas. 2008. A hybrid approach for movie recommendation. Multimedia tools and applications 36 (2008), 55–70.
[28]
Rong Li, Yongcheng Dong, Qifan Kuang, Yiming Wu, Yizhou Li, Min Zhu, and Menglong Li. 2015. Inductive matrix completion for predicting adverse drug reactions (ADRs) integrating drug–target interactions. Chemometrics and Intelligent Laboratory Systems 144 (2015), 71 – 79.
[29]
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. 2023. GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks. In Proceedings of the ACM Web Conference 2023. 417–428.
[30]
Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. 2019. Generating personalized recipes from historical user preferences. arXiv preprint arXiv:1909.00105 (2019).
[31]
Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. 2010. Spectral Regularization Algorithms for Learning Large Incomplete Matrices. Journal of Machine Learning Research 11, 80 (2010), 2287–2322. http://jmlr.org/papers/v11/mazumder10a.html
[32]
Nagarajan Natarajan and Inderjit S. Dhillon. 2014. Inductive matrix completion for predicting gene and disease associations. Bioinformatics 30, 12 (06 2014), i60–i68. https://doi.org/10.1093/bioinformatics/btu269
[33]
Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects. In Conference on Empirical Methods in Natural Language Processing.
[34]
D.A. Nix and A.S. Weigend. 1994. Estimating the mean and variance of the target probability distribution. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), Vol. 1. 55–60 vol.1. https://doi.org/10.1109/ICNN.1994.374138
[35]
Michael P O’Mahony, Neil J Hurley, and Guénolé CM Silvestre. 2006. Detecting noise in recommender system databases. In Proceedings of the 11th international conference on Intelligent user interfaces. 109–115.
[36]
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. 2019. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems 32 (2019).
[37]
Georgina Peake and Jun Wang. 2018. Explanation mining: Post hoc interpretability of latent factor models for recommendation systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2060–2069.
[38]
Benjamin Recht. 2011. A Simpler Approach to Matrix Completion. Journal of Machine Learning Research 12, null (Dec. 2011), 3413–3430.
[39]
Seema Safar, Babita Roslind Jose, Jimson Mathew, and T Santhanakrishnan. 2022. Light-weight Recommendation System using Graph Neural Networks. In 2022 IEEE 19th India Council International Conference (INDICON). IEEE, 1–6.
[40]
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2017. Modeling Relational Data with Graph Convolutional Networks. arxiv:1703.06103 [stat.ML]
[41]
Ohad Shamir and Shai Shalev-Shwartz. 2011. Collaborative Filtering with the Trace Norm: Learning, Bounding, and Transducing. In Proceedings of the 24th Annual Conference on Learning Theory(Proceedings of Machine Learning Research, Vol. 19). PMLR, 661–678.
[42]
Ohad Shamir and Shai Shalev-Shwartz. 2014. Matrix Completion with the Trace Norm: Learning, Bounding, and Transducing. Journal of Machine Learning Research 15 (2014), 3401–3423. http://jmlr.org/papers/v15/shamir14a.html
[43]
Nathan Srebro. 2004. Learning with matrix factorizations. (2004).
[44]
Nathan Srebro and Adi Shraibman. 2005. Rank, trace-norm and max-norm. In Learning Theory: 18th Annual Conference on Learning Theory, COLT 2005, Bertinoro, Italy, June 27-30, 2005. Proceedings 18. Springer, 545–560.
[45]
Raciel Yera Toledo, Yailé Caballero Mota, and Luis Martínez. 2015. Correcting noisy ratings in collaborative recommender systems. Knowledge-Based Systems 76 (2015), 96–108. https://doi.org/10.1016/j.knosys.2014.12.011
[46]
Vojtěch Vančura and Pavel Kordík. 2021. Deep Variational Autoencoder with Shallow Parallel Path for Top-N Recommendation (VASP). In Artificial Neural Networks and Machine Learning – ICANN 2021, Igor Farkaš, Paolo Masulli, Sebastian Otte, and Stefan Wermter (Eds.). Springer International Publishing, Cham, 138–149.
[47]
Vojtěch Vančura, Rodrigo Alves, Petr Kasalický, and Pavel Kordík. 2022. Scalable Linear Shallow Autoencoder for Collaborative Filtering. In Proceedings of the 16th ACM Conference on Recommender Systems (Seattle, WA, USA) (RecSys ’22). Association for Computing Machinery, New York, NY, USA, 604–609. https://doi.org/10.1145/3523227.3551482
[48]
John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. 2009. Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization. In Advances in Neural Information Processing Systems 22, Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta (Eds.). Curran Associates, Inc., 2080–2088.
[49]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?arxiv:1810.00826 [cs.LG]
[50]
Miao Xu, Rong Jin, and Zhi-Hua Zhou. 2013. Speedup Matrix Completion with Side Information: Application to Multi-Label Learning. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (Lake Tahoe, Nevada) (NIPS’13). Curran Associates Inc., Red Hook, NY, USA, 2301–2309.
[51]
Siyong Xu, Cheng Yang, Chuan Shi, Yuan Fang, Yuxin Guo, Tianchi Yang, Luhao Zhang, and Maodi Hu. 2021. Topic-Aware Heterogeneous Graph Neural Network for Link Prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM ’21). Association for Computing Machinery, New York, NY, USA, 2261–2270. https://doi.org/10.1145/3459637.3482485
[52]
Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. 2018. Hierarchical Graph Representation Learning with Differentiable Pooling. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). Vol. 31. Curran Associates, Inc.
[53]
Muhan Zhang and Yixin Chen. 2019. Inductive Graph Pattern Learning for Recommender Systems Based on a Graph Neural Network. CoRR abs/1904.12058 (2019). arXiv:1904.12058http://arxiv.org/abs/1904.12058
[54]
Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An End-to-End Deep Learning Architecture for Graph Classification. In AAAI Conference on Artificial Intelligence.
[55]
Kai Zhong, Prateek Jain, and Inderjit S. Dhillon. 2015. Efficient Matrix Sensing Using Rank-1 Gaussian Measurements. In Algorithmic Learning Theory, Kamalika Chaudhuri, CLAUDIO GENTILE, and Sandra Zilles (Eds.). Springer International Publishing, Cham, 3–18.
[56]
Kai Zhong, Prateek Song, Zhao and, and Inderjit S Dhillon. 2019. Provable Non-linear Inductive Matrix Completion. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alche-Buc, E. Fox, and R. Garnett (Eds.). 11439–11449.
[57]
Lijun Zhou, Antoine Ledent, Qintao Hu, Ting Liu, Jianlin Zhang, and Marius Kloft. 2021. Model uncertainty guides visual object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3581–3589.
[58]
Feng Zhu, Yan Wang, Chaochao Chen, Guanfeng Liu, and Xiaolin Zheng. 2020. A Graphical and Attentional Framework for Dual-Target Cross-Domain Recommendation. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020. 3001–3008.
Index Terms
- Uncertainty-adjusted Inductive Matrix Completion with Graph Neural Networks
Recommendations
Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization
Recovering a large matrix from a small subset of its entries is a challenging problem arising in many real applications, such as image inpainting and recommender systems. Many existing approaches formulate this problem as a general low-rank matrix ...
A non-convex algorithm framework based on DC programming and DCA for matrix completion
Matrix completion aims to recover an unknown low-rank or approximately low-rank matrix from a sampling set of its entries. It is shown that this problem can be solved via its tightest convex relaxation obtained by minimizing the nuclear norm instead of ...
An efficient matrix bi-factorization alternative optimization method for low-rank matrix recovery and completion
In recent years, matrix rank minimization problems have aroused considerable interests from machine learning, data mining and computer vision communities. All of these problems can be solved via their convex relaxations which minimize the trace norm ...
Comments
Information & Contributors
Information
Published In
September 2023
1406 pages
ISBN:9798400702419
DOI:10.1145/3604915
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
- SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
- SIGAI: ACM Special Interest Group on Artificial Intelligence
- SIGKDD: ACM Special Interest Group on Knowledge Discovery in Data
- SIGIR: ACM Special Interest Group on Information Retrieval
- SIGCHI: ACM Special Interest Group on Computer-Human Interaction
- SIGecom: Special Interest Group on Economics and Computation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 September 2023
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Funding Sources
- Ministry of Education - Singapore
Conference
RecSys '23: Seventeenth ACM Conference on Recommender Systems
September 18 - 22, 2023
Singapore, Singapore
Acceptance Rates
Overall Acceptance Rate 254 of 1,295 submissions, 20%
Upcoming Conference
RecSys '24
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 158Total Downloads
- Downloads (Last 12 months)106
- Downloads (Last 6 weeks)8
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format