Abstract
Sequential recommender systems aim to model users’ changing interests based on their historical behavior and predict what they will be interested in at the next moment. In recent years, approaches to modeling users’ long-term/short-term preferences have achieved promising results. Previous works typically model historical interactions through an end-to-end neural network incorporating rich side information, which relies on a final loss function to optimize all parameters. However, they tend to concatenate side information and item ID into a vector representation, leading to irreversible fusion. We propose a two-stage sequence recommendation framework to address this problem. The first stage aims to enhance the representation ability of sequence through a non-invasive bidirectional self-attentive item embedding. In the second stage, we use a time-interval aware Gated Recurrent Units with attention to capture the user’s latest intents, while predicting long-term preferences based on the first stage. To integrate the long-term/short-term preferences, we generate the final preference representation using an attention-based adaptive fusion module. We conduct extensive experiments on four benchmark datasets and the results demonstrate the effectiveness of our proposed model.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
An, M., Wu, F., Wu, C., & et al. (2019). Neural news recommendation with long-and short-term user representations. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 336–345). https://doi.org/10.18653/v1/P19-1033
Cho, K., Van Merriënboer, B., Gulcehre, C., & et al. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078.
Devlin, J., Chang, M. W., Lee, K., & et al. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805.
Devooght, R., & Bersini, H. (2017). Long and short-term recommendations with recurrent neural networks. In Proceedings of the 25th conference on user modeling, adaptation and personalization (pp. 13–21). https://doi.org/10.1145/3079628.3079670
Donkers, T., Loepp, B., & Ziegler, J. (2017). Sequential user-based recurrent neural network recommendations. In Proceedings of the eleventh ACM conference on recommender systems (pp. 152–160). https://doi.org/10.1145/3109859.3109877
Gehring, J., Auli, M., Grangier, D., & et al. (2017). Convolutional sequence to sequence learning. In International conference on machine learning, PMLR (pp. 1243–1252). https://doi.org/10.48550/arXiv.1705.03122
He, Y., Zhang, Y., Liu, W., & et al. (2020). Consistency-aware recommendation for user-generated item list continuation. In Proceedings of the 13th international conference on web search and data mining (pp. 250–258). https://doi.org/10.1145/3336191.3371776
Hidasi, B., & Karatzoglou, A. (2018). Recurrent neural networks with top-k gains for session-based recommendations. In Proceedings of the 27th ACM international conference on information and knowledge management (pp. 843–852). https://doi.org/10.1145/3269206.3271761
Hidasi, B., Karatzoglou, A., Baltrunas, L., & et al. (2015). Session-based recommendations with recurrent neural networks. arXiv:1511.06939.
Hidasi, B., Quadrana, M., Karatzoglou, A., & et al. (2016). Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 241–248). https://doi.org/10.1145/2959100.2959167
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., & et al. (2018). Learning deep representations by mutual information estimation and maximization. arXiv:1808.06670.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Huang, J., Ren, Z., Zhao, W.X., & et al. (2019). Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining (pp. 573–581). https://doi.org/10.1145/3289600.3290972
Kang, W. C., & McAuley, J. (2018). Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM) (pp. 197–206). https://doi.org/10.1109/ICDM.2018.00035
Kong, L., d’Autume, C.D.M., Ling, W., & et al. (2019). A mutual information maximization perspective of language representation learning. arXiv:1910.08350.
Li, J., Ren, P., Chen, Z., & et al. (2017). Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management (pp. 1419–1428). https://doi.org/10.1145/3132847.3132926
Li, W., Saigo, H., Tong, B., & et al. (2021). Topic modeling for sequential documents based on hybrid inter-document topic dependency. Journal of Intelligent Information Systems, 56 (3), 435–458. https://doi.org/10.1007/s10844-020-00635-4.
Li, Z., Lei, C., Zou, P., & et al. (2020). Attention with long-term interval-based gated recurrent units for modeling sequential user behaviors. In International conference on database systems for advanced applications (pp. 663–670). https://doi.org/10.1007/978-3-030-59410-7_44
Li, Z., Zhao, H., Liu, Q., & et al. (2018). Learning from history and present: Next-item recommendation via discriminatively exploiting user behaviors. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1734–1743). https://doi.org/10.1145/3219819.3220014
Liu, C., Li, X., & Cai, G. (2021). Non-invasive self-attention for side information fusion in sequential recommendation. arXiv:2103.03578.
Neil, D., Pfeiffer, M., & Liu, S.C. (2016). Phased lstm: Accelerating recurrent network training for long or event-based sequences. Advances in Neural Information Processing Systems, 29. https://doi.org/10.48550/arXiv.1610.09513.
Quadrana, M., Karatzoglou, A., Hidasi, B., & et al. (2017). Personalizing session-based recommendations with hierarchical recurrent neural networks. In Proceedings of the eleventh ACM conference on recommender systems (pp. 130–137). https://doi.org/10.1145/3109859.3109896
Rendle, S., Freudenthaler, C., Schmidt-Thieme, L., & et al. (2010). Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World Wide Web (pp. 811–820). https://doi.org/10.1145/1772690.1772773
Stratigi, M., Pitoura, E., Nummenmaa, J., & et al. (2022). Sequential group recommendations based on satisfaction and disagreement scores. Journal of Intelligent Information Systems, 58 (2), 227–254. https://doi.org/10.1007/s10844-021-00652-x.
Sun, F., Liu, J., Wu, J., & et al. (2019). Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management (pp. 1441–1450). https://doi.org/10.1145/3357384.3357895
Vassøy, B., Ruocco, M., de Souza da Silva, E., & et al. (2019). Time is of the essence: a joint hierarchical rnn and point process model for time and item predictions. In Proceedings of the twelfth ACM international conference on Web search and data mining (pp. 591–599). https://doi.org/10.1145/3289600.3290987
Wu, C.Y., Ahmed, A., Beutel, A., & et al. (2017). Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining (pp. 495–503). https://doi.org/10.1145/3018661.3018689
Xie, X., Sun, F., Liu, Z., & et al. (2020). Contrastive learning for sequential recommendation. arXiv:2010.14395.
Xu, C., Zhao, P., Liu, Y., & et al. (2019). Graph contextualized self-attention network for session-based recommendation. In IJCAI (Vol. 19 pp. 3940–3946). https://doi.org/10.24963/ijcai.2019/547
Yu, F., Liu, Q., Wu, S., & et al. (2016). A dynamic recurrent model for next basket recommendation. In Proceedings of the 39th International ACM SIGIR conference on research and development in information retrieval. (pp. 729–732) https://doi.org/10.1145/2911451.2914683
Zhang, T., Zhao, P., Liu, Y., & et al. (2019). Feature-level deeper self-attention network for sequential recommendation. In IJCAI (pp. 4320–4326). https://doi.org/10.24963/ijcai.2019/600
Zhou, C., Bai, J., Song, J., & et al. (2018). Atrank: An attention-based user behavior modeling framework for recommendation. In Thirty-Second AAAI conference on artificial intelligence. https://doi.org/10.48550/arXiv.1711.06632
Zhou, K., Wang, H., Zhao, W.X., & et al. (2020). S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM international conference on information & knowledge management (pp. 1893–1902). https://doi.org/10.1145/3340531.3411954
Zhu, Y., Li, H., Liao, Y., & et al. (2017). What to do next: Modeling user behaviors by time-lstm. In IJCAI (Vol. 17 pp. 3602–3608). https://doi.org/10.24963/ijcai.2017/504
Acknowledgements
This work is supported by Natural Science Foundation of China (No.61672337, 61972357) and Medical Science and Technology Project of Zhejiang Province (No.2022KY104).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Data availability
Amazon Beauty, Books and MovieLens are openly available dataset and can be downloaded from their official website.
1. https://nijianmo.github.io/amazon/index.html
2. https://grouplens.org/datasets/movielens/
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lei, J., Li, Y., Yang, S. et al. Two-stage sequential recommendation for side information fusion and long-term and short-term preferences modeling. J Intell Inf Syst 59, 657–677 (2022). https://doi.org/10.1007/s10844-022-00723-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-022-00723-7