research-article

Sequential Recommendation with Graph Neural Networks

Authors:

Yong LiAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 378 - 387

https://doi.org/10.1145/3404835.3462968

Published: 11 July 2021 Publication History

Abstract

Sequential recommendation aims to leverage users' historical behaviors to predict their next interaction. Existing works have not yet addressed two main challenges in sequential recommendation. First, user behaviors in their rich historical sequences are often implicit and noisy preference signals, they cannot sufficiently reflect users' actual preferences. In addition, users' dynamic preferences often change rapidly over time, and hence it is difficult to capture user patterns in their historical sequences. In this work, we propose a graph neural network model called SURGE (short forSeqUential Recommendation with Graph neural nEtworks) to address these two issues. Specifically, SURGE integrates different types of preferences in long-term user behaviors into clusters in the graph by re-constructing loose item sequences into tight item-item interest graphs based on metric learning. This helps explicitly distinguish users' core interests, by forming dense clusters in the interest graph. Then, we perform cluster-aware and query-aware graph convolutional propagation and graph pooling on the constructed graph. It dynamically fuses and extracts users' current activated core interests from noisy user behavior sequences. We conduct extensive experiments on both public and proprietary industrial datasets. Experimental results demonstrate significant performance gains of our proposed method compared to state-of-the-art methods. Further studies on sequence length confirm that our method can model long behavioral sequences effectively and efficiently.

References

[1]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.

[2]

Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. In KDD workshop.

[3]

Jianxin Chang, Chen Gao, Xiangnan He, Depeng Jin, and Yong Li. 2020. Bundle recommendation with graph convolutional networks. In SIGIR. 1673--1676.

[4]

Yu Chen, Lingfei Wu, and Mohammed J Zaki. 2019. Reinforcement learning based graph-to-sequence model for natural question generation. In ICLR.

[5]

Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2020. Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings. In NeurIPS.

[6]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. WWW. 417--426.

[7]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249--256.

[8]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR.

[9]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173--182.

[10]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In ICLR.

[11]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[12]

Bowen Jin, Chen Gao, Xiangnan He, Depeng Jin, and Yong Li. 2020. Multi-behavior recommendation with graph convolutional networks. In SIGIR.

[13]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. 197--206.

[14]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.

[15]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.

[16]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009).

Digital Library

[17]

Junhyun Lee, Inyeop Lee, and Jaewoo Kang. 2019. Self-Attention Graph Pooling. In ICML.

[18]

Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. 2018. Adaptive Graph Convolutional Neural Networks. In AAAI.

[19]

Will Norcliffe-Brown, Stathis Vafeias, and Sarah Parisot. 2018. Learning Conditioned Graph Structures for Interpretable Visual Question Answering. In NeurIPS.

[20]

Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In KDD. 2671--2679.

[21]

Ruihong Qiu, Jingjing Li, Zi Huang, and Hongzhi Yin. 2019. Rethinking the item order in session-based recommendation with graph neural networks. In CIKM. 579--588.

[22]

Ekagra Ranjan, Soumya Sanyal, and Partha Talukdar. 2020. Asap: Adaptive structure aware pooling for learning hierarchical graph representations. In AAAI. 5470--5477.

[23]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW. 811--820.

[24]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WWW. 565--573.

[25]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. 5998--6008.

[26]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.

[27]

Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019 c. Knowledge graph convolutional networks for recommender systems. In WWW. 3307--3313.

[28]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019 a. KGAT: Knowledge Graph Attention Network for Recommendation. In KDD. 950--958.

[29]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019 b. Neural graph collaborative filtering. In SIGIR. 165--174.

[30]

Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020 b. AM-GCN: Adaptive Multi-channel Graph Convolutional Networks. In KDD.

[31]

Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xian-Ling Mao, and Minghui Qiu. 2020 a. Global context enhanced graph neural networks for session-based recommendation. In SIGIR. 169--178.

[32]

Le Wu, Peijie Sun, Yanjie Fu, Richang Hong, Xiting Wang, and Meng Wang. 2019 a. A neural influence diffusion model for social recommendation. In SIGIR. 235--244.

[33]

Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019 b. Session-based recommendation with graph neural networks. In AAAI. 346--353.

[34]

Xuan Wu, Lingxiao Zhao, and Leman Akoglu. 2018. A Quest for Structure: Jointly Learning the Graph Structure and Semi-Supervised Classification. In CIKM.

[35]

Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. 2019. Graph Contextualized Self-Attention Network for Session-based Recommendation. In IJCAI. 3940--3946.

[36]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018a. Graph convolutional neural networks for web-scale recommender systems. In KDD. 974--983.

[37]

Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L Hamilton, and Jure Leskovec. 2018b. Hierarchical Graph Representation Learning with Differentiable Pooling. In NeurIPS.

[38]

Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, and Yiming Yang. 2020. Graph-Revised Convolutional Network. In ECML PKDD.

[39]

Zeping Yu, Jianxun Lian, Ahmad Mahmoody, Gongshen Liu, and Xing Xie. 2019. Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation. In IJCAI. 4213--4219.

[40]

Jun Zhang, Chen Gao, Depeng Jin, and Yong Li. 2021. Group-Buying Recommendation for Social E-Commerce. ICDE.

[41]

Xiang Zhang and Marinka Zitnik. 2020. GNNGuard: Defending Graph Neural Networks against Adversarial Attacks. In NeurIPS.

[42]

Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guojie Song, and Yanfang Ye. 2021 b. Heterogeneous Graph Structure Learning for Graph Neural Networks. In AAAI.

[43]

Tong Zhao, Yozen Liu, Leonardo Neves, Oliver Woodford, Meng Jiang, and Neil Shah. 2021 a. Data Augmentation for Graph Neural Networks. In AAAI.

[44]

Wei Zhao, Benyou Wang, Jianbo Ye, Yongqiang Gao, Min Yang, and Xiaojun Chen. 2018. PLASTIC: Prioritize Long and Short-term Information in Top-n Recommendation using Adversarial Training. In IJCAI. 3676--3682.

[45]

Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In AAAI. 5941--5948.

[46]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD. 1059--1068.

[47]

Han Zhu, Daqing Chang, Ziru Xu, Pengye Zhang, Xiang Li, Jie He, Han Li, Jian Xu, and Kun Gai. 2019. Joint optimization of tree-based index and deep model for recommender systems. In NeurIPS. 3971--3980.

[48]

Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2021. Deep Graph Structure Learning for Robust Representations: A Survey. In IJCAI.

Cited By

Hua QZhou JZhang FDong CXu D(2025)Attention-Enhanced and Knowledge-Fused Dual Item Representations Network for RecommendationTsinghua Science and Technology10.26599/TST.2023.901014330:2(585-599)Online publication date: Apr-2025
https://doi.org/10.26599/TST.2023.9010143
Zhang JLi CZhao Z(2025)Lightweight yet Efficient: An External Attentive Graph Convolutional Network with Positional Prompts for Sequential RecommendationACM Transactions on Information Systems10.1145/3719343Online publication date: 27-Feb-2025
https://doi.org/10.1145/3719343
Wang WLin YRen PChen ZMine TZhao JZhao QZhang MBen XLi Y(2025)Privacy-Preserving Sequential Recommendation with Collaborative ConfusionACM Transactions on Information Systems10.1145/370720443:2(1-25)Online publication date: 18-Jan-2025
https://dl.acm.org/doi/10.1145/3707204
Show More Cited By

Index Terms

Sequential Recommendation with Graph Neural Networks
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Improving Sequential Recommendation with Attribute-Augmented Graph Neural Networks
Advances in Knowledge Discovery and Data Mining
Abstract
Many practical recommender systems provide item recommendation for different users only via mining user-item interactions but totally ignoring the rich attribute information of items that users interact with. In this paper, we propose an attribute-...
Sirius: Sequential Recommendation with Feature Augmented Graph Neural Networks
Database Systems for Advanced Applications
Abstract
Many practical recommender systems recommend personalized items for different users by mining user-item interaction sequences. The interaction sequences, as a whole, imply the manifold collaborative relations among users and items. Further, from ...
Self-adaptive Graph Neural Networks for Personalized Sequential Recommendation
Neural Information Processing
Abstract
Sequential recommendation systems have attracted much attention for the practical applications, and various methods have been proposed. Existing methods based on graph neural networks (GNNs) mostly capture the sequential dependencies on an item ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

The National Key Research and Development Program of China
Beijing National Research Center for Information Science and Technology
Beijing Natural Science Foundation
research fund of Tsinghua University - Tencent Joint Laboratory for Internet Innovation Technology
National Natural Science Foundation of China

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

231
Total Citations
View Citations
3,641
Total Downloads

Downloads (Last 12 months)742
Downloads (Last 6 weeks)42

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hua QZhou JZhang FDong CXu D(2025)Attention-Enhanced and Knowledge-Fused Dual Item Representations Network for RecommendationTsinghua Science and Technology10.26599/TST.2023.901014330:2(585-599)Online publication date: Apr-2025
https://doi.org/10.26599/TST.2023.9010143
Zhang JLi CZhao Z(2025)Lightweight yet Efficient: An External Attentive Graph Convolutional Network with Positional Prompts for Sequential RecommendationACM Transactions on Information Systems10.1145/3719343Online publication date: 27-Feb-2025
https://doi.org/10.1145/3719343
Wang WLin YRen PChen ZMine TZhao JZhao QZhang MBen XLi Y(2025)Privacy-Preserving Sequential Recommendation with Collaborative ConfusionACM Transactions on Information Systems10.1145/370720443:2(1-25)Online publication date: 18-Jan-2025
https://dl.acm.org/doi/10.1145/3707204
Liu MZhang SLong CNejdl WAuer SCha MMoens MNajork M(2025)Facet-Aware Multi-Head Mixture-of-Experts Model for Sequential RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703552(127-135)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703552
Niu JZhou WLuo FZhang YZeng JWen J(2025)Intent-guided Bilateral Long and Short-Term Information Mining with Contrastive Learning for Sequential RecommendationIEEE Transactions on Services Computing10.1109/TSC.2024.3520868(1-15)Online publication date: 2025
https://doi.org/10.1109/TSC.2024.3520868
Zhao ZWang PWang XWen HXie XZhou ZZhang QWang Y(2025)Delayed Bottlenecking: Alleviating Forgetting in Pre-trained Graph Neural NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.351619237:3(1140-1153)Online publication date: Mar-2025
https://doi.org/10.1109/TKDE.2024.3516192
Shi CYan SZhang SWang HLin K(2025)Knowledge-Guided Semantically Consistent Contrastive Learning for sequential recommendationNeural Networks10.1016/j.neunet.2025.107191185(107191)Online publication date: May-2025
https://doi.org/10.1016/j.neunet.2025.107191
Cheng YZheng JWu BMa Q(2025)Sequential recommendation via agent-based irrelevancy skippingNeural Networks10.1016/j.neunet.2025.107134185(107134)Online publication date: May-2025
https://doi.org/10.1016/j.neunet.2025.107134
Li CXie TYu CHu BLi ZCheng LKong BNiu D(2025)DGT: Unbiased sequential recommendation via Disentangled Graph TransformerKnowledge-Based Systems10.1016/j.knosys.2024.112946310(112946)Online publication date: Feb-2025
https://doi.org/10.1016/j.knosys.2024.112946
Liang SKong QLei YLi C(2025)Graphical contrastive learning for multi-interest sequential recommendationExpert Systems with Applications10.1016/j.eswa.2024.125285259(125285)Online publication date: Jan-2025
https://doi.org/10.1016/j.eswa.2024.125285
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten