Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Discrete Federated Multi-behavior Recommendation for Privacy-Preserving Heterogeneous One-Class Collaborative Filtering

Published: 29 April 2024 Publication History

Abstract

Recently, federated recommendation has become a research hotspot mainly because of users’ awareness of privacy in data. As a recent and important recommendation problem, in heterogeneous one-class collaborative filtering (HOCCF), each user may involve of two different types of implicit feedback, that is, examinations and purchases. So far, privacy-preserving HOCCF has received relatively little attention. Existing federated recommendation works often overlook the fact that some privacy sensitive behaviors such as purchases should be collected to ensure the basic business imperatives in e-commerce for example. Hence, the user privacy constraints can and should be relaxed while deploying a recommendation system in real scenarios. In this article, we study the federated multi-behavior recommendation problem under the assumption that purchase behaviors can be collected. Moreover, there are two additional challenges that need to be addressed when deploying federated recommendation. One is the low storage capacity for users’ devices to store all the item vectors, and the other is the low computational power for users to participate in federated learning. To release the potential of privacy-preserving HOCCF, we propose a novel framework, named discrete federated multi-behavior recommendation (DFMR), which allows the collection of the business necessary behaviors (i.e., purchases) by the server. As to reduce the storage overhead, we use discrete hashing techniques, which can compress the parameters down to 1.56% of the real-valued parameters. To further improve the computation-efficiency, we design a memorization strategy in the cache updating module to accelerate the training process. Extensive experiments on four public datasets show the superiority of our DFMR in terms of both accuracy and efficiency.

References

[1]
Muhammad Ammad-ud-din, Elena Ivannikova, Suleiman A. Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. 2019. Federated collaborative filtering for privacy-preserving personalized recommendation system. arXiv:1901.09888. Retrieved from https://arxiv.org/abs/1901.09888
[2]
Vito Walter Anelli, Yashar Deldjoo, Tommaso Di Noia, Antonio Ferrara, and Fedelucio Narducci. 2021. FedeRank: User controlled feedback with federated recommender systems. In Proceedings of the 43rd European Conference on IR Research, Vol. 12656, 32–47.
[3]
Vito Walter Anelli, Yashar Deldjoo, Tommaso Di Noia, Antonio Ferrara, and Fedelucio Narducci. 2021. How to put users in control of their data in federated Top-N recommendation with learning to rank. In Proceedings of the 36th ACM/SIGAPP Symposium on Applied Computing. 1359–1362.
[4]
Alberto Blanco-Justicia, David Sánchez, Josep Domingo-Ferrer, and Krishnamurty Muralidhar. 2023. A critical review on the use (and misuse) of differential privacy in machine learning. Computing Surveys 55, 8 (2023), 160:1–160:16.
[5]
Di Chai, Leye Wang, Kai Chen, and Qiang Yang. 2021. Secure federated matrix factorization. IEEE Intelligent Systems 36, 5 (2021), 11–20.
[6]
Di Chai, Leye Wang, Junxue Zhang, Liu Yang, Shuowei Cai, Kai Chen, and Qiang Yang. 2022. Practical lossless federated singular vector decomposition over billion-scale data. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 46–55.
[7]
Chong Chen, Fei Sun, Min Zhang, and Bolin Ding. 2022. Recommendation unlearning. In Proceedings of the ACM Web Conference 2022. 2768–2777.
[8]
Xiancong Chen, Lin Li, Weike Pan, and Zhong Ming. 2020. A survey on heterogeneous one-class collaborative filtering. ACM Transactions on Information Systems 38, 4 (2020), 35:1–35:54.
[9]
Anders P. K. Dalskov, Daniel Escudero, and Ariel Nof. 2022. Fast fully secure multi-party computation over any ring with two-thirds honest majority. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 653–666.
[10]
Mukund Deshpande and George Karypis. 2004. Item-based Top-N recommendation algorithms. ACM Transactions on Information Systems 22, 1 (2004), 143–177.
[11]
Jingtao Ding, Guanghui Yu, Xiangnan He, Yuhan Quan, Yong Li, Tat-Seng Chua, Depeng Jin, and Jiajie Yu. 2018. Improving implicit recommender systems with view data. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3343–3349.
[12]
Davood Zaman Farsa and Shahryar Rahnamayan. 2020. Discrete Coordinate Descent (DCD). In Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics. 184–190.
[13]
Chen Gao, Xiangnan He, Dahua Gan, Xiangning Chen, Fuli Feng, Yong Li, Tat-Seng Chua, and Depeng Jin. 2019. Neural multi-task recommendation from multi-behavior data. In Proceedings of the 35th IEEE International Conference on Data Engineering. 1554–1557.
[14]
Jialiang Han, Yun Ma, Qiaozhu Mei, and Xuanzhe Liu. 2021. DeepRec: On-device deep learning for privacy-preserving sequential recommendation in mobile commerce. In Proceedings of the Web Conference 2021. 900–911.
[15]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. ACM, 639–648.
[16]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. ACM, 173–182.
[17]
Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 549–558.
[18]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations.
[19]
Bowen Jin, Chen Gao, Xiangnan He, Depeng Jin, and Yong Li. 2020. Multi-behavior recommendation with graph convolutional networks. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 659–668.
[20]
Johnson and Christopher C. 2014. Logistic matrix factorization for implicit feedback data. Advances in Neural Information Processing Systems 27, 78 (2014), 1–9.
[21]
Seiya Jumonji, Kazuya Sakai, Min-Te Sun, and Wei-Shinn Ku. 2023. Privacy-preserving collaborative filtering using fully homomorphic encryption. IEEE Transactions on Knowledge and Data Engineering 35, 3 (2023), 2961–2974.
[22]
Santosh Kabbur, Xia Ning, and George Karypis. 2013. FISM: Factored item similarity models for Top-N recommender systems. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 659–667.
[23]
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista A. Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaïd Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Kone.ny, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrede Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Ozgur, Rasmus Pagh, Hang Qi, Daniel Ramage, Ramesh Raskar, Mariana Raykova, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramer, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. 2021. Advances and open problems in federated learning. Foundations and Trends in Machine Learning 14, 1–2 (2021), 1–210.
[24]
Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv:1610.05492. Retrieved from https://arxiv.org/abs/1610.05492
[25]
Daliang Li and Junpu Wang. 2019. FedMD: Heterogenous federated learning via model distillation. arXiv:1910.03581. Retrieved from https://arxiv.org/abs/1910.03581
[26]
Yaliang Li, Bolin Ding, and Jingren Zhou. 2022. A practical introduction to federated learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 4802–4803.
[27]
Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. ACM, 689–698.
[28]
Feng Liang, Weike Pan, and Zhong Ming. FedRec++: Lossless federated recommendation with explicit feedback. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 4224–4231.
[29]
Feng Liang, Enyue Yang, Weike Pan, Qiang Yang, and Zhong Ming. 2022. Survey of recommender systems based on federated learning. Scientia Sinica Informationis 52, 5 (2022), 713–741.
[30]
Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Dongxiao Yu, Jun Ma, Maarten de Rijke, and Xiuzhen Cheng. 2020. Meta matrix factorization for federated rating predictions. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 981–990.
[31]
Zhaohao Lin, Weike Pan, and Zhong Ming. 2021. FR-FMSS: Federated recommendation via fake marks and secret sharing. In Proceedings of the 15th ACM Conference on Recommender Systems. 668–673.
[32]
Zhaohao Lin, Weike Pan, Qiang Yang, and Zhong Ming. 2022. A generic federated recommendation framework via fake marks and secret sharing. ACM Transactions on Information Systems 41, 2 (2022), 1–37.
[33]
Huitong Lu, Xiaolong Deng, and Junwen Lu. 2023. Research on efficient multi-behavior recommendation method fused with graph neural network. Electronics 12, 9 (2023), 2106.
[34]
Li Ma, Zheng Chen, Yingxun Fu, and Yang Li. 2022. Heterogeneous graph neural network for multi-behavior feature-interaction recommendation. In Proceedings of the 31st International Conference on Artificial Neural Networks, Vol. 13532, 101–112.
[35]
Wanqi Ma, Xiancong Chen, Weike Pan, and Zhong Ming. 2022. VAE++: Variational AutoEncoder for heterogeneous one-class collaborative filtering. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. ACM, 666–674.
[36]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Vol. 54. 1273–1282.
[37]
Lorenzo Minto, Moritz Haller, Benjamin Livshits, and Hamed Haddadi. 2021. Stronger privacy for federated collaborative filtering with implicit feedback. In Proceedings of the 15th ACM Conference on Recommender Systems. 342–350.
[38]
Nicole Mitchell, Johannes Ballé, Zachary Charles, and Jakub Konečný. 2022. Optimizing the communication-accuracy trade-off in federated learning with rate-distortion theory. arXiv:2201.02664. Retrieved from https://arxiv.org/abs/2201.02664
[39]
Chaoyue Niu, Fan Wu, Shaojie Tang, Lifeng Hua, Rongfei Jia, Chengfei Lv, Zhihua Wu, and Guihai Chen. 2020. Billion-scale federated learning on mobile clients: A submodel design with tunable privacy. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 31:1–31:14.
[40]
Rong Pan, Yunhong Zhou, Bin Cao, Nathan Nan Liu, Rajan M. Lukose, Martin Scholz, and Qiang Yang. 2008. One-class collaborative filtering. In Proceedings of the 8th IEEE International Conference on Data Mining. 502–511.
[41]
Weike Pan, Qiang Yang, Wanling Cai, Yaofeng Chen, Qing Zhang, Xiaogang Peng, and Zhong Ming. 2019. Transfer to rank for heterogeneous one-class collaborative filtering. ACM Transactions on Information Systems 37, 1 (2019), 10:1–10:20.
[42]
Weike Pan, Hao Zhong, Congfu Xu, and Zhong Ming. 2015. Adaptive Bayesian personalized ranking for heterogeneous implicit feedbacks. Knowledge-Based Systems 73 (2015), 173–180.
[43]
Mirko Polato. 2021. Federated variational autoencoder for collaborative filtering. In Proceedings of the International Joint Conference on Neural Networks. 1–8.
[44]
Mirko Polato, Alberto Gallinaro, and Fabio Aiolli. 2021. Privacy-preserving kernel computation for vertically partitioned data. In Proceedings of the 29th European Symposium on Artificial Neural Networks.
[45]
Qianzhen Rao, Yang Liu, Weike Pan, and Zhong Ming. 2023. BVAE: Behavior-aware variational autoencoder for multi-behavior multi-task recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems.
[46]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 452–461.
[47]
Ruslan Salakhutdinov and Andriy Mnih. 2007. Probabilistic matrix factorization. In Proceedings of the 21st Annual Conference on Neural Information Processing Systems. 1257–1264.
[48]
Badrul Munir Sarwar, George Karypis, Joseph A. Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International World Wide Web Conference. ACM, 285–295.
[49]
István Selek, Joni Vasara, and Enso Ikonen. 2022. Generalized orthogonalization: A unified framework for gram-schmidt orthogonalization, SVD and PCA. In Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics. 1754–1759.
[50]
Ben Tan, Bo Liu, Vincent W. Zheng, and Qiang Yang. 2020. A federated recommender system for online services. In Proceedings of the 14th ACM Conference on Recommender Systems. 579–581.
[51]
Tamir Tassa and Alon Ben Horin. 2022. Privacy-preserving collaborative filtering by distributed mediation. ACM Transactions on Intelligent Systems and Technology 13, 6 (2022), 102:1–102:26.
[52]
Yu Tian, Jianxin Chang, Yanan Niu, Yang Song, and Chenliang Li. 2022. When multi-level meets multi-interest: A multi-grained neural model for sequential recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1632–1641.
[53]
Chuhan Wu, Fangzhao Wu, Ruixuan Liu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. 2022. Communication efficient federated learning via knowledge distillation. Nature Communications 13 (2022), 0232.
[54]
Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Tao Qi, Yongfeng Huang, and Xing Xie. 2022. A federated graph neural network framework for privacy-preserving personalization. Nature Communications 13, 1 (2022), 3091.
[55]
Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-N recommender systems. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining. ACM, 153–162.
[56]
Yiqing Wu, Ruobing Xie, Yongchun Zhu, Xiang Ao, Xin Chen, Xu Zhang, Fuzhen Zhuang, Leyu Lin, and Qing He. 2022. Multi-view multi-behavior contrastive learning in recommendation. In Proceedings of the 27th Database Systems for Advanced Applications, Vol. 13246. 166–182.
[57]
Lianghao Xia, Yong Xu, Chao Huang, Peng Dai, and Liefeng Bo. 2021. Graph meta network for multi-behavior recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 757–766.
[58]
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology 10, 2 (2019), 12:1–12:19.
[59]
Senci Ying. 2020. Shared MF: A privacy-preserving recommendation system. arXiv:2008.07759. Retrieved from https://arxiv.org/abs/2008.07759
[60]
Wei Yuan, Hongzhi Yin, Fangzhao Wu, Shijie Zhang, Tieke He, and Hao Wang. 2023. Federated unlearning for on-device recommendation. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining. 393–401.
[61]
Honglei Zhang, Fangyuan Luo, Jun Wu, Xiangnan He, and Yidong Li. 2022. LightFR: Lightweight federated recommendation with privacy-preserving matrix factorization. ACM Transactions on Information Systems 41, 4 (2022), 90:1--90:28.
[62]
Hanwang Zhang, Fumin Shen, Wei Liu, Xiangnan He, Huanbo Luan, and Tat-Seng Chua. 2016. Discrete collaborative filtering. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 325–334.
[63]
Zhiwei Zhang, Qifan Wang, Lingyun Ruan, and Luo Si. 2014. Preference preserving hashing for efficient recommendation. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. 183–192.
[64]
Ke Zhou and Hongyuan Zha. 2012. Learning binary codes for collaborative filtering. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 498–506.
[65]
Lei Zhu, Xu Lu, Zhiyong Cheng, Jingjing Li, and Huaxiang Zhang. 2020. Deep collaborative multi-view hashing for large-scale image search. IEEE Transactions on Image Processing 29 (2020), 4643–4655.

Index Terms

  1. Discrete Federated Multi-behavior Recommendation for Privacy-Preserving Heterogeneous One-Class Collaborative Filtering

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Information Systems
    ACM Transactions on Information Systems  Volume 42, Issue 5
    September 2024
    809 pages
    EISSN:1558-2868
    DOI:10.1145/3618083
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 29 April 2024
    Online AM: 18 March 2024
    Accepted: 06 March 2024
    Revised: 30 December 2023
    Received: 18 March 2023
    Published in TOIS Volume 42, Issue 5

    Check for updates

    Author Tags

    1. Federated recommendation
    2. heterogeneous implicit feedback
    3. discrete hashing

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • Guangdong Basic and Applied Basic Research Foundation
    • National Key Research and Development Program of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 584
      Total Downloads
    • Downloads (Last 12 months)584
    • Downloads (Last 6 weeks)72
    Reflects downloads up to 11 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media