EFVAE: Efficient Federated Variational Autoencoder for Collaborative Filtering
Authors: 
Lu Zhang, Qian Rong, Xuanang Ding, Guohui Li, Ling YuanAuthors Info & Claims
Lu Zhang, Qian Rong, Xuanang Ding, Guohui Li, Ling YuanAuthors Info & ClaimsPages 3176 - 3185
Abstract
Federated recommender systems are used to address privacy issues in recommendations. Among them, FedVAE extends the representative non-linear recommendation method MultVAE. However, the bottleneck of FedVAE lies in its communication load during training, as the parameter volume of its first and last layers is correlated with the number of items. This leads to significant communication cost during the model's transmission phases (distribution and upload), making FedVAE's implementation extremely challenging. To address these challenges, we propose an Efficient Federated Variational AutoEncoder for collaborative filtering, EFVAE, which core is the Federated Collaborative Importance Sampling (FCIS) method. FCIS reduces communication costs through a client-to-server collaborative sampling mechanism and provides satisfactory recommendation performance through dynamic multi-stage approximation of the decoding distribution. Extensive experiments and analyses on real-world datasets confirm that EFVAE significantly reduces communication costs by up to 94.51% while maintaining the recommendation performance. Moreover, its recommendation performance is better on sparse datasets, with improvements reaching up to 13.79%.
References
[1]
Muhammad Ammad-Ud-Din, Elena Ivannikova, Suleiman A Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. 2019. Federated collaborative filtering for privacy-preserving personalized recommendation system. arXiv preprint arXiv:1901.09888 (2019).
[2]
Artem Babenko and Victor Lempitsky. 2015. The Inverted Multi-Index. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, 6 (June 2015), 1247--1260.
[3]
Yoshua Bengio and Jean-Sébastien Senecal. 2003. Quick Training of Probabilistic Neural Nets by Importance Sampling. In AISTATS Workshop (Proceedings of Machine Learning Research, Vol. R4). 17--24.
[4]
Di Chai, Leye Wang, Kai Chen, and Qiang Yang. 2020. Secure federated matrix factorization. IEEE Intelligent Systems, Vol. 36, 5 (2020), 11--20.
[5]
Jin Chen, Defu Lian, Binbin Jin, Xu Huang, Kai Zheng, and Enhong Chen. 2022. Fast variational autoencoder with inverted multi-index for collaborative filtering. In Proceedings of the ACM Web Conference 2022. 1944--1954.
[6]
Yucheng Chen, Chenyuan Feng, and Daquan Feng. 2023. Privacy-preserving hierarchical federated recommendation systems. IEEE Communications Letters (2023).
[7]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7--10.
[8]
Woo-Seok Choi, Matthew Tomei, Jose Rodrigo Sanchez Vicarte, Pavan Kumar Hanumolu, and Rakesh Kumar. 2018. Guaranteeing local differential privacy on ultra-low-power systems. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA). IEEE, 561--574.
[9]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191--198.
[10]
Adrian Flanagan, Were Oyomno, Alexander Grigorievskiy, Kuan E Tan, Suleiman A Khan, and Muhammad Ammad-Ud-Din. 2021. Federated multi-view matrix factorization for personalized recommendations. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14--18, 2020, Proceedings, Part II. Springer, 324--347.
[11]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249--256.
[12]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19--25, 2017, Carles Sierra (Ed.). ijcai.org, 1725--1731. https://doi.org/10.24963/IJCAI.2017/239
[13]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[14]
Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14--16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.6114
[15]
Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 world wide web conference. 689--698.
[16]
Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Dongxiao Yu, Jun Ma, Maarten de Rijke, and Xiuzhen Cheng. 2020. Meta matrix factorization for federated rating predictions. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 981--990.
[17]
Zhaohao Lin, Weike Pan, Qiang Yang, and Zhong Ming. 2022. A generic federated recommendation framework via fake marks and secret sharing. ACM Transactions on Information Systems, Vol. 41, 2 (2022), 1--37.
[18]
Ruixuan Liu, Yang Cao, Yanlin Wang, Lingjuan Lyu, Yun Chen, and Hong Chen. 2023. PrivateRec: Differentially Private Model Training and Online Serving for Federated News Recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4539--4548.
[19]
Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019. Learning disentangled representations for recommendation. Advances in neural information processing systems, Vol. 32 (2019).
[20]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273--1282.
[21]
Khalil Muhammad, Qinqin Wang, Diarmuid O'Reilly-Morgan, Elias Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor. 2020. Fedfast: Going beyond average for faster training of federated recommender systems. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 1234--1242.
[22]
Vasileios Perifanis, George Drosatos, Giorgos Stamatelatos, and Pavlos S Efraimidis. 2023. FedPOIRec: Privacy-preserving federated poi recommendation with social influence. Information Sciences, Vol. 623 (2023), 767--790.
[23]
Vasileios Perifanis and Pavlos S Efraimidis. 2022. Federated neural collaborative filtering. Knowledge-Based Systems, Vol. 242 (2022), 108441.
[24]
Mirko Polato. 2021. Federated variational autoencoder for collaborative filtering. In 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--8.
[25]
Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang, and Xing Xie. 2020. Privacy-preserving news recommendation model learning. arXiv preprint arXiv:2003.09592 (2020).
[26]
Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoc Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. 2023. Semi-decentralized federated ego graph learning for recommendation. In Proceedings of the ACM Web Conference 2023. 339--348.
[27]
Noveen Sachdeva, Giuseppe Manco, Ettore Ritacco, and Vikram Pudi. 2019. Sequential variational autoencoders for collaborative filtering. In Proceedings of the twelfth ACM international conference on web search and data mining. 600--608.
[28]
Adi Shamir. 1979. How to Share a Secret. Commun. ACM, Vol. 22, 11 (1979), 612--613.
[29]
Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I Nikolenko. 2020. Recvae: A new variational autoencoder for top-n recommendations with implicit feedback. In Proceedings of the 13th international conference on web search and data mining. 528--536.
[30]
Quoc-Tuan Truong, Aghiles Salah, and Hady W Lauw. 2021. Bilateral variational autoencoder for collaborative filtering. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 292--300.
[31]
Qinyong Wang, Hongzhi Yin, Tong Chen, Junliang Yu, Alexander Zhou, and Xiangliang Zhang. 2022. Fast-adapting and privacy-preserving federated recommender system. The VLDB Journal, Vol. 31, 5 (2022), 877--896.
[32]
Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Tao Qi, Yongfeng Huang, and Xing Xie. 2022. A federated graph neural network framework for privacy-preserving personalization. Nature Communications, Vol. 13, 1 (2022), 3091.
[33]
Liu Yang, Ben Tan, Vincent W Zheng, Kai Chen, and Qiang Yang. 2020. Federated recommendation systems. Federated Learning: Privacy and Incentive (2020), 225--239.
[34]
Jingwei Yi, Fangzhao Wu, Chuhan Wu, Ruixuan Liu, Guangzhong Sun, and Xing Xie. 2021. Efficient-FedRec: Efficient federated learning framework for privacy-preserving news recommendation. arXiv preprint arXiv:2109.05446 (2021).
[35]
Senci Ying. 2020. Shared MF: A privacy-preserving recommendation system. arXiv preprint arXiv:2008.07759 (2020).
[36]
Sergej Zerr, Stefan Siersdorfer, Jonathon Hare, and Elena Demidova. 2012. Privacy-aware image classification and search. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 35--44.
[37]
Lu Zhang, Guohui Li, Ling Yuan, Xuanang Ding, and Qian Rong. 2024. HN3S: A Federated AutoEncoder framework for Collaborative Filtering via Hybrid Negative Sampling and Secret Sharing. Information Processing & Management, Vol. 61, 2 (2024), 103580. https://doi.org/10.1016/j.ipm.2023.103580
Index Terms
- EFVAE: Efficient Federated Variational Autoencoder for Collaborative Filtering
Recommendations
Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender SystemsUser-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
Typicality-Based Collaborative Filtering Recommendation
Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...
Exploiting user interests for collaborative filtering: interests expansion via personalized ranking
CIKM '10: Proceedings of the 19th ACM international conference on Information and knowledge managementIn real applications, a given user buys or rates an item based on his/her interests. Learning to leverage this interest information is often critical for recommender systems. However, in existing recommender systems, the information about latent user ...
Comments
Information & Contributors
Information
Published In
October 2024
5705 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
CIKM '24
Sponsor:
CIKM '24: The 33rd ACM International Conference on Information and Knowledge Management
October 21 - 25, 2024
ID, Boise, USA
Acceptance Rates
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 122Total Downloads
- Downloads (Last 12 months)122
- Downloads (Last 6 weeks)45
Reflects downloads up to 22 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in