Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3477495.3531910acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
short-paper

Exploiting Session Information in BERT-based Session-aware Sequential Recommendation

Published: 07 July 2022 Publication History

Abstract

In recommendation systems, utilizing the user interaction history as sequential information has resulted in great performance improvement. However, in many online services, user interactions are commonly grouped by sessions that presumably share preferences, which requires a different approach from ordinary sequence representation techniques. To this end, sequence representation models with a hierarchical structure or various viewpoints have been developed but with a rather complex network structure. In this paper, we propose three methods to improve recommendation performance by exploiting session information while minimizing additional parameters in a BERT-based sequential recommendation model: using session tokens, adding session segment embeddings, and a time-aware self-attention. We demonstrate the feasibility of the proposed methods through experiments on widely used recommendation datasets.

Supplementary Material

MP4 File (SIGIR22-sp2138.mp4)
Presentation of the paper, "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation" by the first author, Jinseok Seol.

References

[1]
Yukuo Cen, Jianwei Zhang, Xu Zou, Chang Zhou, Hongxia Yang, and Jie Tang. 2020. Controllable multi-interest framework for recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2942--2951.
[2]
Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong. 2022. Intent Contrastive Learning for Sequential Recommendation. arXiv preprint arXiv:2202.02519 (2022).
[3]
Junsu Cho, SeongKu Kang, Dongmin Hyun, and Hwanjo Yu. 2021. Unsupervised Proxy Selection for Session-based Recommender Systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 327--336.
[4]
Sung Min Cho, Eunhyeok Park, and Sungjoo Yoo. 2020. MEANTIME: Mixture of attention mechanisms with multi-temporal embeddings for sequential recommendation. In Fourteenth ACM Conference on Recommender Systems . 515--520.
[5]
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364 (2017).
[6]
Qiang Cui, Shu Wu, Yan Huang, and Liang Wang. 2019. A hierarchical contextual attention-based network for sequential recommendation. Neurocomputing, Vol. 358 (2019), 141--149.
[7]
Gabriel de Souza Pereira Moreira, Sara Rabhi, Jeong Min Lee, Ronay Ak, and Even Oldridge. 2021. Transformers4Rec: Bridging the Gap between NLP and Sequential/Session-Based Recommendation. In Fifteenth ACM Conference on Recommender Systems . 143--153.
[8]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[9]
Ziwei Fan, Zhiwei Liu, Yu Wang, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, and Philip S Yu. 2022. Sequential Recommendation via Stochastic Self-Attention. arXiv preprint arXiv:2201.06035 (2022).
[10]
Ziwei Fan, Zhiwei Liu, Jiawei Zhang, Yun Xiong, Lei Zheng, and Philip S Yu. 2021. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management . 433--442.
[11]
Milena Filipovic, Blagoj Mitrevski, Diego Antognini, Emma Lejal Glaude, Boi Faltings, and Claudiu Musat. 2020. Modeling Online Behavior in Recommender Systems: The Importance of Temporal Context. arXiv preprint arXiv:2009.08978 (2020).
[12]
Peiyao Han, Nan Wang, Kun Li, and Xiaokun Li. 2021. CASR: A Collaborative Attention Model for Session-based Recommendation. In 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 290--296.
[13]
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 (2020).
[14]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173--182.
[15]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
[16]
Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. 2018b. Improving sequential recommendation with knowledge-enhanced memory networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 505--514.
[17]
Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, and Changsheng Xu. 2018a. Csan: Contextual self-attention network for user sequential recommendation. In Proceedings of the 26th ACM international conference on Multimedia. 447--455.
[18]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 197--206.
[19]
Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining . 1748--1757.
[20]
Sara Latifi, Noemi Mauro, and Dietmar Jannach. 2021. Session-aware recommendation: A surprising quest for the state-of-the-art. Information Sciences, Vol. 573 (2021), 291--315.
[21]
Chenyi Lei, Shouling Ji, and Zhao Li. 2019. Tissa: A time slice self-attention approach for modeling sequential user behaviors. In The World Wide Web Conference. 2964--2970.
[22]
Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1419--1428.
[23]
Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time interval aware self-attention for sequential recommendation. In Proceedings of the 13th international conference on web search and data mining . 322--330.
[24]
Yang Li, Tong Chen, Peng-Fei Zhang, and Hongzhi Yin. 2021. Lightweight self-attentive sequential recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management . 967--977.
[25]
Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 2018. STAMP: short-term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1831--1839.
[26]
Ilya Loshchilov and Frank Hutter. 2018. Fixing weight decay regularization in adam. (2018).
[27]
Chen Ma, Peng Kang, and Xue Liu. 2019. Hierarchical gating networks for sequential recommendation. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining . 825--833.
[28]
Yifei Ma, Balakrishnan Narayanaswamy, Haibin Lin, and Hao Ding. 2020. Temporal-contextual recommendation in real-time. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2291--2299.
[29]
Tu Minh Phuong, Tran Cong Thanh, and Ngo Xuan Bach. 2019. Neural session-aware recommendation. IEEE Access, Vol. 7 (2019), 86884--86896.
[30]
Ruihong Qiu, Zi Huang, Tong Chen, and Hongzhi Yin. 2021. Exploiting positional information for session-based recommendation. ACM Transactions on Information Systems (TOIS), Vol. 40, 2 (2021), 1--24.
[31]
Massimo Quadrana, Paolo Cremonesi, and Dietmar Jannach. 2018. Sequence-aware recommender systems. ACM Computing Surveys (CSUR), Vol. 51, 4 (2018), 1--36.
[32]
Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In proceedings of the Eleventh ACM Conference on Recommender Systems. 130--137.
[33]
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019).
[34]
Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2019. Repeatnet: A repeat aware neural recommendation machine for session-based recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 4806--4813.
[35]
Ruiyang Ren, Zhaoyang Liu, Yaliang Li, Wayne Xin Zhao, Hui Wang, Bolin Ding, and Ji-Rong Wen. 2020. Sequential recommendation with self-attentive multi-adversarial network. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 89--98.
[36]
Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. 285--295.
[37]
Mostafa Shabani, Dat Thanh Tran, Martin Magris, Juho Kanniainen, and Alexandros Iosifidis. 2022. Multi-head Temporal Attention-Augmented Bilinear Network for Financial time series prediction. arXiv preprint arXiv:2201.05459 (2022).
[38]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management . 1441--1450.
[39]
Zhongchuan Sun, Bin Wu, Youwei Wang, and Yangdong Ye. 2022. Sequential Graph Collaborative Filtering. Information Sciences (2022).
[40]
Jiaxi Tang, Francois Belletti, Sagar Jain, Minmin Chen, Alex Beutel, Can Xu, and Ed H. Chi. 2019. Towards neural mixture recommender for long range dependent user sequences. In The World Wide Web Conference. 1782--1793.
[41]
Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the eleventh ACM international conference on web search and data mining . 565--573.
[42]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, Vol. 30 (2017).
[43]
Chenyang Wang, Weizhi Ma, Min Zhang, Chong Chen, Yiqun Liu, and Shaoping Ma. 2020 b. Toward dynamic user intention: Temporal evolutionary effects of item relations in sequential recommendation. ACM Transactions on Information Systems (TOIS), Vol. 39, 2 (2020), 1--33.
[44]
Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. 2020 a. Next-item recommendation with sequential hypergraphs. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 1101--1110.
[45]
Yikai Wang, Liang Zhang, Quanyu Dai, Fuchun Sun, Bo Zhang, Yang He, Weipeng Yan, and Yongjun Bao. 2019. Regularized adversarial sampling and deep time-aware attention for click-through rate prediction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management . 349--358.
[46]
Chen Wu and Ming Yan. 2017. Session-aware information embedding for e-commerce product recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management . 2379--2382.
[47]
Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020. SSE-PT: Sequential recommendation via personalized transformer. In Fourteenth ACM Conference on Recommender Systems. 328--337.
[48]
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 346--353.
[49]
Yongji Wu, Lu Yin, Defu Lian, Mingyang Yin, Neil Zhenqiang Gong, Jingren Zhou, and Hongxia Yang. 2021. Rethinking Lifelong Sequential Recommendation with Incremental Multi-Interest Attention. arXiv preprint arXiv:2105.14060 (2021).
[50]
Chengfeng Xu, Jian Feng, Pengpeng Zhao, Fuzhen Zhuang, Deqing Wang, Yanchi Liu, and Victor S Sheng. 2021. Long-and short-term self-attention network for sequential recommendation. Neurocomputing, Vol. 423 (2021), 580--589.
[51]
Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. 2019. Self-attention with functional time representation learning. Advances in neural information processing systems, Vol. 32 (2019).
[52]
Rui Ye, Qing Zhang, and Hengliang Luo. 2020 b. Cross-Session Aware Temporal Convolutional Network for Session-based Recommendation. In 2020 International Conference on Data Mining Workshops (ICDMW). IEEE, 220--226.
[53]
Wenwen Ye, Shuaiqiang Wang, Xu Chen, Xuepeng Wang, Zheng Qin, and Dawei Yin. 2020 a. Time matters: Sequential recommendation with complex temporal information. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1459--1468.
[54]
Bo Yu, Ruoqian Zhang, Wei Chen, and Junhua Fang. 2021. Graph neural network based model for multi-behavior session-based recommendation. GeoInformatica (2021), 1--19.
[55]
Tianzi Zang, Yanmin Zhu, Jing Zhu, Yanan Xu, and Haobing Liu. 2022. MPAN: Multi-parallel attention network for session-based recommendation. Neurocomputing, Vol. 471 (2022), 230--241.
[56]
Beichuan Zhang, Zhijiao Xiao, and Shenghua Zhong. [n.d.]. Deep Self-Attention for Sequential Recommendation. ( [n.,d.]).
[57]
Lu-ning Zhang, Jian-wei Liu, Zhi-yan Song, and Xin Zuo. 2021 b. Temporal attention augmented transformer Hawkes process. Neural Computing and Applications (2021), 1--15.
[58]
Mengqi Zhang, Shu Wu, Meng Gao, Xin Jiang, Ke Xu, and Liang Wang. 2020 b. Personalized graph neural networks with attention mechanism for session-aware recommendation. IEEE Transactions on Knowledge and Data Engineering (2020).
[59]
Mengqi Zhang, Shu Wu, Xueli Yu, Qiang Liu, and Liang Wang. 2022 b. Dynamic graph neural networks for sequential recommendation. IEEE Transactions on Knowledge and Data Engineering (2022).
[60]
Qian Zhang, Shoujin Wang, Wenpeng Lu, Chong Feng, Xueping Peng, and Qingxiang Wang. 2022 a. Rethinking Adjacent Dependency in Session-based Recommendations. arXiv preprint arXiv:2201.12532 (2022).
[61]
Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, and Xiaofang Zhou. 2019. Feature-level Deeper Self-Attention Network for Sequential Recommendation. In IJCAI . 4320--4326.
[62]
Yuhui Zhang, Hao Ding, Zeren Shui, Yifei Ma, James Zou, Anoop Deoras, and Hao Wang. 2021 a. Language Models as Recommender Systems: Evaluations and Limitations. In I (Still) Can't Believe It's Not Better! NeurIPS 2021 Workshop .
[63]
Yin Zhang, Yun He, Jianling Wang, and James Caverlee. 2020 a. Adaptive hierarchical translation-based sequential recommendation. In Proceedings of The Web Conference 2020. 2984--2990.
[64]
Yujia Zheng, Siyi Liu, and Zailei Zhou. 2019. Balancing multi-level interactions for session-based recommendation. arXiv preprint arXiv:1910.13527 (2019).
[65]
Jing Zhu, Yanan Xu, and Yanmin Zhu. 2020. Modeling Long-Term and Short-Term Interests with Parallel Attentions for Session-Based Recommendation. In International Conference on Database Systems for Advanced Applications. Springer, 654--669.

Cited By

View all
  • (2024)Optimizing Session-Aware Recommenders: A Deep Dive into GRU-Based Latent Interaction IntegrationFuture Internet10.3390/fi1602005116:2(51)Online publication date: 1-Feb-2024
  • (2024)Pre-training general user representation with multi-type APP behaviorsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/612(5535-5544)Online publication date: 3-Aug-2024
  • (2024)FSASA: Sequential recommendation based on fusing session-aware models and self-attention networksComputer Science and Information Systems10.2298/CSIS230522067G21:1(1-20)Online publication date: 2024
  • Show More Cited By

Index Terms

  1. Exploiting Session Information in BERT-based Session-aware Sequential Recommendation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2022
    3569 pages
    ISBN:9781450387323
    DOI:10.1145/3477495
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 July 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. sequential recommendation
    2. session-aware recommendation
    3. temporal self-attention

    Qualifiers

    • Short-paper

    Funding Sources

    • Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT)

    Conference

    SIGIR '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)78
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 11 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Optimizing Session-Aware Recommenders: A Deep Dive into GRU-Based Latent Interaction IntegrationFuture Internet10.3390/fi1602005116:2(51)Online publication date: 1-Feb-2024
    • (2024)Pre-training general user representation with multi-type APP behaviorsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/612(5535-5544)Online publication date: 3-Aug-2024
    • (2024)FSASA: Sequential recommendation based on fusing session-aware models and self-attention networksComputer Science and Information Systems10.2298/CSIS230522067G21:1(1-20)Online publication date: 2024
    • (2024)Learning the Dynamics in Sequential Recommendation by Exploiting Real-time InformationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679955(4288-4292)Online publication date: 21-Oct-2024
    • (2024)TiCoSeRec: Augmenting Data to Uniform Sequences by Time Intervals for Effective RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.332431236:6(2686-2700)Online publication date: Jun-2024
    • (2024)T3SRSExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122260238:PEOnline publication date: 27-Feb-2024
    • (2024)SNRBERT: session-based news recommender using BERTUser Modeling and User-Adapted Interaction10.1007/s11257-024-09409-x34:4(1071-1085)Online publication date: 31-Jul-2024
    • (2023)BERT4Loc: BERT for Location—POI Recommender SystemFuture Internet10.3390/fi1506021315:6(213)Online publication date: 12-Jun-2023
    • (2023)Temporally Dynamic Session-Keyword Aware Sequential Recommendation System2023 IEEE International Conference on Data Mining Workshops (ICDMW)10.1109/ICDMW60847.2023.00027(157-164)Online publication date: 4-Dec-2023
    • (2023)A sequential neural recommendation system exploiting BERT and LSTM on social media postsComplex & Intelligent Systems10.1007/s40747-023-01191-410:1(721-744)Online publication date: 3-Aug-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media