Learning Global and Multi-granularity Local Representation with MLP for Sequential Recommendation
Article No.: 87, Pages 1 - 15
Abstract
Sequential recommendation aims to predict the next item of interest to users based on their historical behavior data. Usually, users’ global and local preferences jointly affect the final recommendation result in different ways. Most existing works use transformers to globally model sequences, which makes them face the dilemma of quadratic computational complexity when dealing with long sequences. Moreover, the scope setting of the user’s local preference is usually static and single, and cannot cover richer multi-level local semantics. To this end, we proposed a parallel architecture for capturing global representation and Multi-granularity Local dependencies with MLP for sequential Recommendation (MLM4Rec). For global representation, we utilize modified MLP-Mixer to capture global information of user sequences due to its simplicity and efficiency. For local representation, we incorporate convolution into MLP and propose a multi-granularity local awareness mechanism for capturing richer local semantic information. Moreover, we introduced a weight pooling method to adaptively fuse local-global representations instead of directly concatenation. Our model has the advantages of low complexity and high efficiency thanks to its simple MLP structure. Experimental results on three public datasets demonstrate the effectiveness of our proposed model. Our code is available here.
References
[1]
Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 355–364.
[2]
Zhankui He, Handong Zhao, Zhe Lin, Zhaowen Wang, Ajinkya Kale, and Julian McAuley. 2021. Locker: Locally constrained self-attentive sequential recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3088–3092.
[3]
Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415 (2016).
[4]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In 4th International Conference on Learning Representations.
[5]
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).
[6]
Juyong Jiang, Jae Boum Kim, Yingtao Luo, Kai Zhang, and Sunghun Kim. 2022. AdaMCT: Adaptive mixture of CNN-transformer for sequential recommendation. arXiv preprint arXiv:2205.08776 (2022).
[7]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining. 197–206.
[8]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations.
[9]
Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1748–1757.
[10]
Hojoon Lee, Dongyoon Hwang, Sunghwan Hong, Changyeon Kim, Seungryong Kim, and Jaegul Choo. 2021. MOI-mixer: Improving MLP-mixer with multi order interactions in sequential recommendation. arXiv preprint arXiv:2108.07505 (2021).
[11]
Muyang Li, Xiangyu Zhao, Chuan Lyu, Minghao Zhao, Runze Wu, and Ruocheng Guo. 2022. MLP4Rec: A pure MLP architecture for sequential recommendations. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. 2138–2144.
[12]
Hanxiao Liu, Zihang Dai, David So, and Quoc V. Le. 2021. Pay attention to MLPs. Advances in Neural Information Processing Systems 34 (2021), 9204–9215.
[13]
Huafeng Liu, Liping Jing, Jingxuan Wen, Zhicheng Wu, Xiaoyi Sun, Jiaqi Wang, Lin Xiao, and Jian Yu. 2020. Deep global and local generative model for recommendation. In Proceedings of The Web Conference 2020. 551–561.
[14]
Jie Lu, Dianshuang Wu, Mingsong Mao, Wei Wang, and Guangquan Zhang. 2015. Recommender system application developments: A survey. Decision Support Systems 74 (2015), 12–32.
[15]
Chen Ma, Peng Kang, and Xue Liu. 2019. Hierarchical gating networks for sequential recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 825–833.
[16]
Steffen Rendle. 2010. Factorization machines. In The 10th IEEE International Conference on Data Mining. 995–1000.
[17]
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web. 811–820.
[18]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1441–1450.
[19]
Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 565–573.
[20]
Yuki Tatsunami and Masato Taki. 2021. RaftMLP: Do MLP-based models dream of winning over computer vision? arXiv preprint arXiv:2108.04384 (2021).
[21]
Ilya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. 2021. MLP-mixer: An all-MLP architecture for vision. NeurIPS 2021, 24261–24272.
[22]
Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, and Hervé Jégou. 2023. ResMLP: Feedforward networks for image classification with data-efficient training. IEEE Trans. Pattern Anal. Mach. Intell. 45, 4 (2023), 5314–5321.
[23]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30. 5998–6008.
[24]
Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z. Sheng, and Mehmet Orgun. 2019. Sequential recommender systems: Challenges, progress and prospects. arXiv preprint arXiv:2001.04830 (2019).
[25]
Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Jiajie Xu, Victor S. Sheng, Zhiming Cui, Xiaofang Zhou, and Hui Xiong. 2019. Recurrent convolutional neural network for sequential recommendation. In The World Wide Web Conference. 3398–3404.
[26]
Tan Yu, Xu Li, Yunfeng Cai, Mingming Sun, and Ping Li. 2022. S2-MLP: Spatial-shift MLP architecture for vision. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 297–306.
[27]
Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M. Jose, and Xiangnan He. 2019. A simple convolutional generative network for next item recommendation. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. 582–590.
[28]
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. Comput. Surveys 52, 1 (2019), 1–38.
[29]
Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, and Xiaofang Zhou. 2019. Feature-level deeper self-attention network for sequential recommendation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. 4320–4326.
[30]
Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. 2022. Filter-enhanced MLP is all you need for sequential recommendation. In Proceedings of the ACM Web Conference 2022. 2388–2399.
Index Terms
- Learning Global and Multi-granularity Local Representation with MLP for Sequential Recommendation
Recommendations
User Popularity Preference Aware Sequential Recommendation
Computational Science – ICCS 2023AbstractIn recommender systems, users’ preferences for item popularity are diverse and dynamic, which reveals the different items that users prefer. Therefore, identifying user popularity preferences are significant for personalized recommendations. ...
Capturing Multi-granularity Interests with Capsule Attentive Network for Sequential Recommendation
Web Information Systems Engineering – WISE 2021AbstractThe sequential recommender system attempts to predict the next interaction based on user’s historical behaviors, which is a challenging problem due to intricate sequential dependencies and user’s various interests underneath the interactions. ...
Multi-interest Diversification for End-to-end Sequential Recommendation
Sequential recommenders capture dynamic aspects of users’ interests by modeling sequential behavior. Previous studies on sequential recommendations mostly aim to identify users’ main recent interests to optimize the recommendation accuracy; they often ...
Comments
Information & Contributors
Information
Published In
May 2024
707 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 February 2024
Online AM: 26 December 2023
Accepted: 15 December 2023
Received: 27 June 2023
Published in TKDD Volume 18, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- NSFC
- Major project of natural science research in Universities of Jiangsu Province
- Suzhou Science and Technology Development Program
- Priority Academic Program Development of Jiangsu Higher Education Institutions and the Exploratory Self-selected Project of the State Key Laboratory of Software Development Environment
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 310Total Downloads
- Downloads (Last 12 months)310
- Downloads (Last 6 weeks)30
Reflects downloads up to 12 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in