research-article

Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling

Authors:

Steven C.H. HoiAuthors Info & Claims

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 359 - 367

https://doi.org/10.1145/3394486.3403078

Published: 20 August 2020 Publication History

Abstract

Sequential user behavior modeling plays a crucial role in online user-oriented services, such as product purchasing, news feed consumption, and online advertising. The performance of sequential modeling heavily depends on the scale and quality of historical behaviors. However, the number of user behaviors inherently follows a long-tailed distribution, which has been seldom explored. In this work, we argue that focusing on tail users could bring more benefits and address the long tails issue by learning transferrable parameters from both optimization and feature perspectives. Specifically, we propose a gradient alignment optimizer and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail. Such methods can also deal with the cold-start problem of new users. Moreover, it could be directly adaptive to various well-established sequential models. Extensive experiments on four real-world datasets verify the superiority of our framework compared with the state-of-the-art baselines.

Supplementary Material

MP4 File (3394486.3403078.mp4)

This video is the introduction of Article: Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling, mainly including background, intuition, Methodology and experiment.

Download
107.57 MB

References

[1]

Alex Beutel, Ed H Chi, Zhiyuan Cheng, Hubert Pham, and John Anderson. 2017. Beyond globally optimal: Focused learning for improved recommendations. In Proceedings of the 26th International Conference on World Wide Web. 203--212.

Digital Library

[2]

Sebastian Flennerhag, Pablo G Moreno, Neil D Lawrence, and Andreas Damianou. 2018. Transferring knowledge across learning processes. arXiv preprint arXiv:1812.01054 (2018).

[3]

Chengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2018. Frage: Frequency-agnostic word representation. In Advances in neural information processing systems. 1334--1345.

[4]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014a. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.

[5]

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014b. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).

[6]

Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 191--200.

[7]

Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 549--558.

Digital Library

[8]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).

[9]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 197--206.

[10]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8 (2009), 30--37.

Digital Library

[11]

Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1419--1428.

Digital Library

[12]

Chenghao Liu, Steven CH Hoi, Peilin Zhao, Jianling Sun, and Ee-Peng Lim. 2016. Online adaptive passive-aggressive methods for non-negative matrix factorization and its applications. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 1161--1170.

Digital Library

[13]

Chenghao Liu, Tao Jin, Steven CH Hoi, Peilin Zhao, and Jianling Sun. 2017. Collaborative topic regression for online recommender systems: an online and Bayesian approach. Machine Learning, Vol. 106, 5 (2017), 651--670.

Digital Library

[14]

Chen Ma, Peng Kang, and Xue Liu. 2019. Hierarchical gating networks for sequential recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 825--833.

Digital Library

[15]

Andriy Mnih and Russ R Salakhutdinov. 2008. Probabilistic matrix factorization. In Advances in neural information processing systems. 1257--1264.

[16]

Alex Nichol, Joshua Achiam, and John Schulman. 2018. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 (2018).

[17]

James R Norris. 1998. Markov chains. Number 2. Cambridge university press.

[18]

Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 1149--1154.

[19]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web. ACM, 811--820.

Digital Library

[20]

Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. 2018. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11910 (2018).

[21]

Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in neural information processing systems. 2234--2242.

[22]

Robert Sanders. 1987. The Pareto principle: its use and abuse. Journal of Services Marketing (1987).

[23]

Guy Shani, David Heckerman, and Ronen I Brafman. 2005. An MDP-based recommender system. Journal of Machine Learning Research, Vol. 6, Sep (2005), 1265--1295.

Digital Library

[24]

Mohit Sharma and George Karypis. 2019. Adaptive matrix completion for the users and the items in tail. In The World Wide Web Conference. 3223--3229.

Digital Library

[25]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).

[26]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. ACM, 565--573.

Digital Library

[27]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[28]

Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2015. Learning hierarchical representation model for nextbasket recommendation. In Proceedings of the 38th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 403--412.

Digital Library

[29]

Shoujin Wang, Liang Hu, Longbing Cao, Xiaoshui Huang, Defu Lian, and Wei Liu. 2018a. Attention-based transactional context embedding for next-item recommendation. In Thirty-Second AAAI Conference on Artificial Intelligence .

[30]

Weiqing Wang, Hongzhi Yin, Ling Chen, Yizhou Sun, Shazia Sadiq, and Xiaofang Zhou. 2017. ST-SAGE: A spatial-temporal sparse additive generative model for spatial item recommendation. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 8, 3 (2017), 1--25.

Digital Library

[31]

Weiqing Wang, Hongzhi Yin, Xingzhong Du, Quoc Viet Hung Nguyen, and Xiaofang Zhou. 2018b. Tpm: A temporal personalized model for spatial item recommendation. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 9, 6 (2018), 1--25.

Digital Library

[32]

Jianwen Yin, Chenghao Liu, Jundong Li, BingTian Dai, Yun-chen Chen, Min Wu, and Jianling Sun. 2019. Online Collaborative Filtering with Implicit Feedback. In International Conference on Database Systems for Advanced Applications. Springer, 433--448.

[33]

Lu Yu, Chuxu Zhang, Shangsong Liang, and Xiangliang Zhang. 2019. Multi-order Attentive Ranking Model for Sequential Recommendation. (2019).

[34]

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2018. A Simple but Hard-to-Beat Baseline for Session-based Recommendations. arXiv preprint arXiv:1808.05163 (2018).

[35]

Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E Hinton. 2019 a. Lookahead Optimizer: k steps forward, 1 step back. In Advances in Neural Information Processing Systems. 9593--9604.

[36]

Shuai Zhang, Yi Tay, Lina Yao, Aixin Sun, and Jake An. 2019 b. Next Item Recommendation with Self-Attentive Metric Learning. (2019).

[37]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059--1068.

Digital Library

[38]

Andrew Zimdars, David Maxwell Chickering, and Christopher Meek. 2001. Using temporal data for making recommendations. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 580--588.

Cited By

Yu RChen YZhao MYu JXu TYu MLi X(2024)Debiasing Recommenders Through Personalized Popularity-Aware MarginsICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447541(6255-6259)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447541
Lv ZWang FZhang SZhang WKuang KWu F(2024)Parameters Efficient Fine-Tuning for Long-Tailed Sequential RecommendationArtificial Intelligence10.1007/978-981-99-8850-1_36(442-459)Online publication date: 4-Feb-2024
https://doi.org/10.1007/978-981-99-8850-1_36
Kim KHyun DYun SPark CChen HDuh WHuang HKato MMothe JPoblete B(2023)MELT: Mutual Enhancement of Long-Tailed User and Item for Sequential RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591725(68-77)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591725
Show More Cited By

Index Terms

Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Long-Tailed Instance Segmentation Using Gumbel Optimized Loss
Computer Vision – ECCV 2022
Abstract
Major advancements have been made in the field of object detection and segmentation recently. However, when it comes to rare categories, the state-of-the-art methods fail to detect them, resulting in a significant performance gap between rare and ...
Decoupled Contrastive Learning for Long-Tailed Distribution
Pattern Recognition and Computer Vision
Abstract
Self-supervised contrastive learning is popularly used to obtain powerful representation models. However, unlabeled data in the real world naturally exhibits a long-tailed distribution, making the traditional instance-wise contrastive learning ...
Imbalance fault diagnosis under long-tailed distribution: Challenges, solutions and prospects
Abstract
Intelligent fault diagnosis based on deep learning has yielded remarkable progress for its strong feature representation capability in recent years. Nevertheless, in engineering scenarios, machines often operate under normal condition ...
Highlights
- Define the class long-tailed distribution in intelligent fault diagnosis for the first time.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

3664 pages

ISBN:9781450379984

DOI:10.1145/3394486

General Chairs:
Rajesh Gupta
UC San Diego, USA
,
Yan Liu
USC, USA
,
Program Chairs:
Mohak Shah
LG Electronics, USA
,
Suju Rajan
Linkedin, USA
,
Publications Chairs:
Jiliang Tang
Michigan State, USA
,
B. Aditya Prakash
Georgia Tech, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

The National Research Foundation Singapore, and AI Singapore under its Research Programme

Conference

KDD '20

Sponsor:

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

July 6 - 10, 2020

CA, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
1,092
Total Downloads

Downloads (Last 12 months)45
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu RChen YZhao MYu JXu TYu MLi X(2024)Debiasing Recommenders Through Personalized Popularity-Aware MarginsICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447541(6255-6259)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447541
Lv ZWang FZhang SZhang WKuang KWu F(2024)Parameters Efficient Fine-Tuning for Long-Tailed Sequential RecommendationArtificial Intelligence10.1007/978-981-99-8850-1_36(442-459)Online publication date: 4-Feb-2024
https://doi.org/10.1007/978-981-99-8850-1_36
Kim KHyun DYun SPark CChen HDuh WHuang HKato MMothe JPoblete B(2023)MELT: Mutual Enhancement of Long-Tailed User and Item for Sequential RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591725(68-77)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591725
Lu YNakamura KIchise R(2023)HyperRS: Hypernetwork-Based Recommender System for the User Cold-Start ProblemIEEE Access10.1109/ACCESS.2023.323639111(5453-5463)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3236391
Cheng MLiu QZhang WLiu ZZhao HChen E(2023)A general tail item representation enhancement framework for sequential recommendationFrontiers of Computer Science10.1007/s11704-023-3112-y18:6Online publication date: 28-Dec-2023
https://doi.org/10.1007/s11704-023-3112-y
Liu ZMao QLiu CFang YSun J(2022)On Size-Oriented Long-Tailed Graph Classification of Graph Neural NetworksProceedings of the ACM Web Conference 202210.1145/3485447.3512197(1506-1516)Online publication date: 25-Apr-2022
https://doi.org/10.1145/3485447.3512197
Chen YLiu ZLi JMcAuley JXiong C(2022)Intent Contrastive Learning for Sequential RecommendationProceedings of the ACM Web Conference 202210.1145/3485447.3512090(2172-2182)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512090
Wu LHe XWang XZhang KWang M(2022)A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.3145690(1-1)Online publication date: 2022
https://doi.org/10.1109/TKDE.2022.3145690
Ren WWang LLiu KGuo RPeng LFu Y(2022)Mitigating Popularity Bias in Recommendation with Unbalanced Interactions: A Gradient Perspective2022 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM54844.2022.00054(438-447)Online publication date: Nov-2022
https://doi.org/10.1109/ICDM54844.2022.00054
Li YChen HSun XSun ZLi LCui LYu PXu GDemartini GZuccon GCulpepper JHuang ZTong H(2021)Hyperbolic Hypergraphs for Sequential RecommendationProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482351(988-997)Online publication date: 30-Oct-2021
https://doi.org/10.1145/3459637.3482351

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents