research-article

Multi-Behavior Sequential Recommendation With Temporal Graph Transformer

Authors:

Jian PeiAuthors Info & Claims

IEEE Transactions on Knowledge and Data Engineering, Volume 35, Issue 6

Pages 6099 - 6112

https://doi.org/10.1109/TKDE.2022.3175094

Published: 01 June 2023 Publication History

Abstract

Modeling time-evolving preferences of users with their sequential item interactions, has attracted increasing attention in many online applications. Hence, sequential recommender systems have been developed to learn the dynamic user interests from the historical interactions for suggesting items. However, the interaction pattern encoding functions in most existing sequential recommender systems have focused on single type of user-item interactions. In many real-life online platforms, user-item interactive behaviors are often multi-typed (e.g., click, add-to-favorite, purchase) with complex cross-type behavior inter-dependencies. Learning from informative representations of users and items based on their multi-typed interaction data, is of great importance to accurately characterize the time-evolving user preference. In this work, we tackle the dynamic user-item relation learning with the awareness of multi-behavior interactive patterns. Towards this end, we propose a new <underline>T</underline>emporal <underline>G</underline>raph <underline>T</underline>ransformer (TGT) recommendation framework to jointly capture dynamic short-term and long-range user-item interactive patterns, by exploring the evolving correlations across different types of behaviors. The new TGT method endows the sequential recommendation architecture to distill dedicated knowledge for type-specific behavior relational context and the implicit behavior dependencies. Experiments on the real-world datasets indicate that our method TGT consistently outperforms various state-of-the-art recommendation methods. Our model implementation codes are available at <uri>https://github.com/akaxlh/TGT</uri>.

References

[1]

Y. Cao, X. Wang, X. He, Z. Hu, and T.-S. Chua, “Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences,” in Proc. World Wide Web Conf., 2019, pp. 151–161.

Digital Library

[2]

C. Chen, M. Zhang, W. Ma, Y. Liu, and S. Ma, “Jointly non-sampling learning for knowledge graph enhanced recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 189–198.

[3]

T. Chen, H. Yin, H. Chen, R. Yan, Q. V. H. Nguyen, and X. Li, “AIR: Attentional intention-aware recommender systems,” in Proc. IEEE Int. Conf. Data Eng., 2019, pp. 304–315.

[4]

S. Fan et al., “Metapath-guided heterogeneous graph neural network for intent recommendation,” in Proc. Int. Conf. Knowl. Discov. Data Mining, 2019, pp. 2478–2486.

[5]

W. Fan et al., “Graph neural networks for social recommendation,” in Proc. World Wide Web Conf., 2019, pp. 417–426.

[6]

H. Fang, D. Zhang, Y. Shu, and G. Guo, “Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations,” Trans. Inf. Syst., vol. 39, no. 1, pp. 1–42, 2020.

Digital Library

[7]

C. Gao et al., “Neural multi-task recommendation from multi-behavior data,” in Proc. Int. Conf. Data Eng., 2019, pp. 1554–1557.

[8]

H. Guo et al., “DeepFM: A factorization-machine based neural network for CTR prediction,” in Proc. Int. Joint Conf. Artif. Intell., 2017, pp. 1725–1731.

[9]

C. Hansen et al., “Contextual and sequential user embeddings for large-scale music recommendation,” in Proc. Int. Conf. Recommender Syst., 2020, pp. 53–62.

[10]

R. He and J. McAuley, “Fusing similarity models with Markov chains for sparse sequential recommendation,” in Proc. IEEE Int. Conf. Data Mining, 2016, pp. 191–200.

[11]

X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “LightGCN: Simplifying and powering graph convolution network for recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 639–648.

[12]

X. He et al., “Neural collaborative filtering,” in Proc. 26th Int. Conf. World Wide Web, 2017, pp. 173–182.

Digital Library

[13]

B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, “Session-based recommendations with recurrent neural networks,” in Proc. Int. Conf. Learn. Representations, 2016.

[14]

Z. Hu, Y. Dong, K. Wang, and Y. Sun, “Heterogeneous graph transformer,” in Proc. Web Conf., 2020, pp. 2704–2710.

Digital Library

[15]

C. Huang, “Recent advances in heterogeneous relation learning for recommendation,” 2021,.

[16]

C. Huang et al., “Graph-enhanced multi-task learning of multi-level transition dynamics for session-based recommendation,” in Proc. AAAI Conf. Artif. Intell., 2021, pp. 4123–4130.

[17]

C. Huang et al., “Online purchase prediction via multi-scale modeling of behavior dynamics,” in Proc. Int. Conf. Knowl. Discov. Data Mining, 2019, pp. 2613–2622.

[18]

C. Huang et al., “Knowledge-aware coupled graph neural network for social recommendation,” in Proc. AAAI Conf. Artif. Intell., 2021.

[19]

J. Huang, W. X. Zhao, H. Dou, J.-R. Wen, and E. Y. Chang, “Improving sequential recommendation with knowledge-enhanced memory networks,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2018, pp. 505–514.

[20]

B. Jin, C. Gao, X. He, D. Jin, and Y. Li, “Multi-behavior recommendation with graph convolutional networks,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 659–668.

[21]

J. Jin et al., “An efficient neighborhood-based interaction model for recommendation on heterogeneous graph,” in Proc. Int. Conf. Knowl. Discov. Data Mining, 2020, pp. 75–84.

[22]

W.-C. Kang and J. McAuley, “Self-attentive sequential recommendation,” in Proc. IEEE Int. Conf. Data Mining, 2018, pp. 197–206.

[23]

J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, “Neural attentive session-based recommendation,” in Proc. Int. Conf. Inf. Knowl. Manage., 2017, pp. 1419–1428.

[24]

Q. Li, Z. Han, and X.-M. Wu, “Deeper insights into graph convolutional networks for semi-supervised learning,” in Proc. Int. Conf. Artif. Intell., 2018, pp. 3538–3545.

[25]

M. Liu, H. Gao, and S. Ji, “Towards deeper graph neural networks,” in Proc. Int. Conf. Knowl. Discov. Data Mining, 2020, pp. 338–348.

[26]

Q. Liu, Y. Zeng, R. Mokhosi, and H. Zhang, “STAMP: Short-term attention/memory priority model for session-based recommendation,” in Int. Conf. Knowl. Discov. Data Mining, 2018, pp. 1831–1839.

[27]

S. Liu, I. Ounis, C. Macdonald, and Z. Meng, “A. heterogeneous graph neural model for cold-start recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 2029–2032.

[28]

C. Ma et al., “Memory augmented graph neural networks for sequential recommendation,” in Proc. Int. Conf. Artif. Intell., 2020, pp. 1798–1802.

[29]

J. Ma, C. Zhou, P. Cui, H. Yang, and W. Zhu, “Learning disentangled representations for recommendation,” in Proc. Neural Inf. Process. Syst., 2019, pp. 5711–5722.

[30]

W. Ma et al., “Jointly learning explainable rules for recommendation with knowledge graph,” in Proc. World Wide Web Conf., 2019, pp. 1210–1221.

[31]

W. Meng et al., “Incorporating user micro-behaviors and item knowledge into multi-task learning for session-based recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 1091–1100.

[32]

M. Quadrana, A. Karatzoglou, B. Hidasi, and P. Cremonesi, “Personalizing session-based recommendations with hierarchical recurrent neural networks,” in Proc. Int. Conf. Recommender Syst., 2017, pp. 130–137.

[33]

S. Rendle et al., “Factorizing personalized markov chains for next-basket recommendation,” in Proc. Int. Conf. World Wide Web, 2010, pp. 811–820.

[34]

S. Rendle et al., “BPR: Bayesian personalized ranking from implicit feedback,” in Proc. Int. Conf. Uncertainty Artif. Intell., 2009, pp. 452–461.

[35]

C. Shi, B. Hu, W. X. Zhao, and S. Y. Philip, “Heterogeneous information network embedding for recommendation,” Trans. Knowl. Data Eng., vol. 31, no. 2, pp. 357–370, 2018.

Digital Library

[36]

F. Sun et al., “BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer,” in Proc. Int. Conf. Inf. Knowl. Manage., 2019, pp. 1441–1450.

[37]

J. Tang and K. Wang, “Personalized top-n sequential recommendation via convolutional sequence embedding,” in Proc. Int. Conf. Web Search Data Mining, 2018, pp. 565–573.

[38]

M. M. Tanjim, C. Su, E. Benjamin, D. Hu, L. Hong, and J. McAuley, “Attentive sequential models of latent intent for next item recommendation,” in Proc. Web Conf., 2020, pp. 2528–2534.

[39]

A. Vaswani et al., “Attention is all you need,” in Proc. Neural Inf. Process. Syst., 2017, pp. 6000–6010.

[40]

C. Wang, M. Zhang, W. Ma, Y. Liu, and S. Ma, “Make it a chorus: Knowledge-and time-aware item modeling for sequential recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 109–118.

[41]

J. Wang, K. Ding, L. Hong, H. Liu, and J. Caverlee, “Next-item recommendation with sequential hypergraphs,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 1101–1110.

[42]

S. Wang et al., “Sequential recommender systems: Challenges, progress and prospects,” in Proc. Int. Joint Conf. Artif. Intell., 2019.

[43]

S. Wang, L. Hu, Y. Wang, Q. Z. Sheng, M. A. Orgun, and L. Cao, “Intention2Basket: A neural intention-driven approach for dynamic next-basket planning,” in Proc. Int. Joint Conf. Artif. Intell., 2020, pp. 2333–2339.

[44]

W. Wang et al., “Beyond clicks: Modeling multi-relational item graph for session-based target behavior prediction,” in Proc. Web Conf., 2020, pp. 3056–3062.

[45]

X. Wang et al., “Neural graph collaborative filtering,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2019, pp. 165–174.

[46]

Y. Wang, S. Tang, Y. Lei, W. Song, S. Wang, and M. Zhang, “DisenHAN: Disentangled heterogeneous graph attention network for recommendation,” in Proc. Int. Conf. Inf. Knowl. Manage., 2020, pp. 1605–1614.

[47]

Z. Wang, G. Lin, H. Tan, Q. Chen, and X. Liu, “CKAN: Collaborative knowledge-aware attentive network for recommender systems,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 219–228.

[48]

Z. Wang, W. Wei, G. Cong, X.-L. Li, X.-L. Mao, and M. Qiu, “Global context enhanced graph neural networks for session-based recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 169–178.

[49]

W. Wei, C. Huang, L. Xia, Y. Xu, J. Zhao, and D. Yin, “Contrastive meta learning with behavior multiplicity for recommendation,” in Proc. Int. Conf. Web Search Data Mining, 2022, pp. 1120–1128.

[50]

J. Wu et al., “Self-supervised graph learning for recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2021, pp. 726–735.

[51]

L. Wu, D. Hu, L. Hong, and H. Liu, “Turning clicks into purchases: Revenue optimization for product search in e-commerce,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2018, pp. 365–374.

[52]

L. Wu, P. Sun, Y. Fu, R. Hong, X. Wang, and M. Wang, “A neural influence diffusion model for social recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2019, pp. 235–244.

[53]

S. Wu et al., “Session-based recommendation with graph neural networks,” in Proc. Int. Conf. Artif. Intell., 2019, pp. 346–353.

[54]

L. Xia, C. Huang, Y. Xu, P. Dai, B. Zhang, and L. Bo, “Multiplex behavioral relation learning for recommendation via memory augmented transformer network,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 2397–2406.

[55]

L. Xia et al., “Knowledge-enhanced hierarchical graph transformer network for multi-behavior recommendation,” in Proc. AAAI Conf. Artif. Intell., 2021, pp. 4486–4493.

[56]

L. Xia, C. Huang, Y. Xu, J. Zhao, D. Yin, and J. X. Huang, “Hypergraph contrastive collaborative filtering,” 2022,.

[57]

L. Xia, Y. Xu, C. Huang, P. Dai, and L. Bo, “Graph meta network for multi-behavior recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2021, pp. 757–766.

[58]

X. Xin, X. He, Y. Zhang, Y. Zhang, and J. Jose, “Relational collaborative filtering: Modeling multiple item relations for recommendation,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2019, pp. 125–134.

[59]

F. Xue, X. He, X. Wang, J. Xu, K. Liu, and R. Hong, “Deep item-based collaborative filtering for top-N recommendation,” Trans. Inf. Syst., vol. 37, no. 3, pp. 1–25, 2019.

Digital Library

[60]

R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, “Graph convolutional neural networks for web-scale recommender systems,” in Proc. Int. Conf. Knowl. Discov. Data Mining, 2018, pp. 974–983.

[61]

M. Zhang, S. Wu, X. Yu, Q. Liu, and L. Wang, “Dynamic graph neural networks for sequential recommendation,” Trans. Knowl. Data Eng., 2022, pp. 378–387.

[62]

C. Zhao, C. Li, R. Xiao, H. Deng, and A. Sun, “CATN: Cross-domain recommendation for cold-start users via aspect transfer network,” in Proc. Int. Conf. Res. Develop. Inf. Retrieval, 2020, pp. 229–238.

[63]

Y. Zheng, C. Gao, X. Li, X. He, Y. Li, and D. Jin, “Disentangling user interest and conformity for recommendation with causal embedding,” in Proc. Web Conf., 2021, pp. 2980–2991.

[64]

K. Zhou et al., “S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization,” in Proc. Int. Conf. Inf. Knowl. Manage., 2020, pp. 1893–1902.

[65]

M. Zhou et al., “Micro behaviors: A new perspective in e-commerce recommender systems,” in Proc. Int. Conf. Web Search Data Mining, 2018, pp. 727–735.

[66]

H. Zhu et al., “Bilinear graph neural network with neighbor interactions,” in Proc. Int. Joint Conf. Artif. Intell., 2020, pp. 1452–1458.

Cited By

Li WLin XPan WMing Z(2024)Dynamic Stage-aware User Interest Learning for Heterogeneous Sequential RecommendationProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688103(465-474)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688103
Li XChen HYu JZhao MXu TZhang WYu MAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Global Heterogeneous Graph and Target Interest Denoising for Multi-behavior Sequential RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635857(387-395)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635857
Yu XLi WZhang CWang JZhao YLiu FPan QLiu HDing JChen D(2024)Time-aware multi-behavior graph network model for complex group behavior predictionInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10366661:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103666
Show More Cited By

Recommendations

Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Learning dynamic user preference has become an increasingly important component for many online platforms (e.g., video-sharing sites, e-commerce systems) to make sequential recommendations. Previous works have made many efforts to model item-item ...
Personalized Behavior-Aware Transformer for Multi-Behavior Sequential Recommendation
MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Sequential Recommendation (SR) captures users' dynamic preferences by modeling how users transit among items. However, SR models that utilize only single type of behavior interaction data encounter performance degradation when the sequences are short. To ...
Improving Sequential Recommendation with Attribute-Augmented Graph Neural Networks
Advances in Knowledge Discovery and Data Mining
Abstract
Many practical recommender systems provide item recommendation for different users only via mining user-item interactions but totally ignoring the rich attribute information of items that users interact with. In this paper, we propose an attribute-...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Knowledge and Data Engineering

IEEE Transactions on Knowledge and Data Engineering Volume 35, Issue 6

June 2023

1074 pages

ISSN:1041-4347

Issue’s Table of Contents

1041-4347 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 01 June 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li WLin XPan WMing Z(2024)Dynamic Stage-aware User Interest Learning for Heterogeneous Sequential RecommendationProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688103(465-474)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688103
Li XChen HYu JZhao MXu TZhang WYu MAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Global Heterogeneous Graph and Target Interest Denoising for Multi-behavior Sequential RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635857(387-395)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635857
Yu XLi WZhang CWang JZhao YLiu FPan QLiu HDing JChen D(2024)Time-aware multi-behavior graph network model for complex group behavior predictionInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10366661:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103666
Rahmani MCaverlee JWang F(2023)Incorporating Time in Sequential Recommendation ModelsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608830(784-790)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608830
Mukande TAli ECaputo ADong RO'Connor NFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)A Flash Attention Transformer for Multi-Behaviour RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615206(4210-4214)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615206
Kou ZManchanda SLin SXie MWang HZhang XFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Modeling Sequential Collaborative User Behaviors For Seller-Aware Next Basket RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614973(1097-1106)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614973
Tomlinson KNeville JYang LWan MLu CSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Workplace Recommendation with Temporal Network ObjectivesProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599932(4958-4969)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599932
Tran VSalha-Galvan GSguerra BHennequin RChen HDuh WHuang HKato MMothe JPoblete B(2023)Attention Mixtures for Time-Aware Sequential RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591951(1821-1826)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591951
Zhou WHuang HShi RSong XLin XWang XJin H(2023)Temporal Heterogeneous Information Network Embedding via Semantic EvolutionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.328726035:12(13031-13042)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3287260

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents