research-article

MCRPL: A Pretrain, Prompt, and Fine-tune Paradigm for Non-overlapping Many-to-one Cross-domain Recommendation

Authors:

Yongqiang Jiang,

Hongzhi YinAuthors Info & Claims

ACM Transactions on Information Systems, Volume 42, Issue 4

Article No.: 97, Pages 1 - 24

https://doi.org/10.1145/3641860

Published: 09 February 2024 Publication History

Abstract

Cross-domain Recommendation is the task that tends to improve the recommendations in the sparse target domain by leveraging the information from other rich domains. Existing methods of cross-domain recommendation mainly focus on overlapping scenarios by assuming users are totally or partially overlapped, which are taken as bridges to connect different domains. However, this assumption does not always hold, since it is illegal to leak users’ identity information to other domains. Conducting Non-overlapping MCR (NMCR) is challenging, since (1) the absence of overlapping information prevents us from directly aligning different domains, and this situation may get worse in the MCR scenario, and (2) the distribution between source and target domains makes it difficult for us to learn common information across domains. To overcome the above challenges, we focus on NMCR and devise MCRPL as our solution. To address Challenge 1, we first learn shared domain-agnostic and domain-dependent prompts and pre-train them in the pre-training stage. To address Challenge 2, we further update the domain-dependent prompts with other parameters kept fixed to transfer the domain knowledge to the target domain. We conduct experiments on five real-world domains, and the results show the advance of our MCRPL method compared with several recent SOTA baselines. Moreover, our source codes have been publicly released.

References

[1]

Nawaf Alharbi and Doina Caragea. 2022. Cross-domain self-attentive sequential recommendations. In Proceedings of the International Conference on Data Science and Applications. 601–614.

[2]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33 (2020), 1877–1901.

[3]

Tong Chen, Hongzhi Yin, Hongxu Chen, Lin Wu, Hao Wang, Xiaofang Zhou, and Xue Li. 2018. Tada: Trend alignment with dual-attention multi-task recurrent neural networks for sales prediction. In Proceedings of the IEEE International Conference on Data Mining. IEEE, 49–58.

[4]

Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong. 2022. Intent contrastive learning for sequential recommendation. In Proceedings of the ACM International World Wide Web Conferences. 491–502.

[5]

Qiang Cui, Tao Wei, Yafeng Zhang, and Qing Zhang. 2020. HeroGRAPH: A heterogeneous graph framework for multi-target cross-domain recommendation. In Proceedings of the ACM Conference on Recommender Systems. 1–7.

[6]

Ignacio Fernández-Tobías, Iván Cantador, Paolo Tomeo, Vito Walter Anelli, and Tommaso Di Noia. 2019. Addressing the user cold start with cross-domain collaborative filtering: Exploiting item metadata in matrix factorization. User Model. User-Adapt. Interact. 29, 2 (2019), 443–486.

Digital Library

[7]

Chen Gao, Xiangning Chen, Fuli Feng, Kai Zhao, Xiangnan He, Yong Li, and Depeng Jin. 2019. Cross-domain recommendation without sharing user-relevant data. In Proceedings of the ACM International World Wide Web Conferences. 491–502.

Digital Library

[8]

Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In Proceedings of the 16th ACM Conference on Recommender Systems. 299–315.

Digital Library

[9]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 249–256.

[10]

Lei Guo, Li Tang, Tong Chen, Lei Zhu, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2021. DA-GCN: A domain-aware attentive graph convolution network for shared-account cross-domain sequential recommendation. In Proceedings of the International Joint Conference on Artificial Intelligence. 2483–2489.

[11]

Lei Guo, Chunxiao Wang, Xinhua Wang, Lei Zhu, and Hongzhi Yin. 2023. Automated prompting for non-overlapping cross-domain sequential recommendation. arXiv:2304.04218 [cs.IR].

[12]

Lei Guo, Jinyu Zhang, Tong Chen, Xinhua Wang, and Hongzhi Yin. 2022. Reinforcement learning-enhanced shared-account cross-domain sequential recommendation. IEEE Transactions on Knowledge and Data Engineering 35, 7 (2023), 7397–7411.

[13]

Lei Guo, Jinyu Zhang, Li Tang, Tong Chen, Lei Zhu, and Hongzhi Yin. 2022. Time interval-enhanced graph neural network for shared-account cross-domain sequential recommendation. IEEE Transactions on Neural Networks and Learning Systems (unpublished).

[14]

Bowen Hao, Chaoqun Yang, Lei Guo, Junliang Yu, and Hongzhi Yin. 2023. Motif-based prompt learning for universal cross-domain recommendation. arXiv:2310.13303 [cs.IR].

[15]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In Proceedings of the International Conference on Learning Representations. 811–820.

[16]

Shuhui Jiang, Zhengming Ding, and Yun Fu. 2019. Heterogeneous recommendation via deep low-rank sparse collective factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 5 (2019), 1097–1111.

[17]

Wang-Cheng Kang and Julian J. McAuley. 2018. Self-attentive sequential recommendation. In Proceedings of the IEEE International Conference on Data Mining. 197–206.

[18]

Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In Proceedings of the International Conference on Knowledge Discovery & Data Mining. 1748–1757.

[19]

Adit Krishnan, Mahashweta Das, Mangesh Bendre, Hao Yang, and Hari Sundaram. 2020. Transfer learning via contextual invariants for one-to-many cross-domain recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 1081–1090.

[20]

Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 3045–3059.

[21]

Bin Li, Qiang Yang, and Xiangyang Xue. 2009. Can movies and books collaborate? Cross-domain collaborative filtering for sparsity reduction. In Proceedings of the International Joint Conference on Artificial Intelligence.

[22]

Bin Li, Qiang Yang, and Xiangyang Xue. 2009. Transfer learning for collaborative filtering via a rating-matrix generative model. In Proceedings of the International Conference on Machine Learning. 617–624.

Digital Library

[23]

Lei Li, Yongfeng Zhang, and Li Chen. 2023. Personalized prompt learning for explainable recommendation. ACM Transactions on Information Systems 41, 4 (2023), 1–26.

Digital Library

[24]

Pan Li and Alexander Tuzhilin. 2020. Ddtcdr: Deep dual transfer cross domain recommendation. In Proceedings of the International Conference on Web Search and Data Mining. 331–339.

Digital Library

[25]

Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the International Joint Conference on Natural Language Processing. 4582–4597.

[26]

Meng Liu, Jianjun Li, Guohui Li, and Peng Pan. 2020. Cross domain recommendation via bi-directional transfer graph collaborative filtering networks. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 885–894.

Digital Library

[27]

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surv. 55, 9 (2023), 1–35.

Digital Library

[28]

Weiming Liu, Jiajie Su, Chaochao Chen, and Xiaolin Zheng. 2021. Leveraging distribution alignment via stein path for cross-domain cold-start recommendation. In Advances in Neural Information Processing Systems. 19223–19234.

[29]

Weiming Liu, Xiaolin Zheng, Mengling Hu, and Chaochao Chen. 2022. Collaborative filtering with attribution alignment for review-based non-overlapped cross domain recommendation. In Proceedings of the ACM International World Wide Web Conferences. 1181–1190.

Digital Library

[30]

Weiming Liu, Xiaolin Zheng, Mengling Hu, and Chaochao Chen. 2022. Collaborative filtering with attribution alignment for review-based non-overlapped cross domain recommendation. In Proceedings of the ACM International World Wide Web Conferences. 1181–1190.

Digital Library

[31]

Muyang Ma, Pengjie Ren, Yujie Lin, Zhumin Chen, Jun Ma, and Maarten de Rijke. 2019. \(\pi\)-Net: A parallel information-sharing network for shared-account cross-domain sequential recommendations. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 685–694.

[32]

Jarana Manotumruksa, Dimitrios Rafailidis, Craig Macdonald, and Iadh Ounis. 2019. On cross-domain transfer in venue recommendation. In Proceedings of the European Conference on Information Retrieval. 443–456.

Digital Library

[33]

Orly Moreno, Bracha Shapira, Lior Rokach, and Guy Shani. 2012. Talmud: Transfer learning for multiple domains. In Proceedings of the ACM International Conference on Information and Knowledge Management. 425–434.

[34]

Quoc Viet Hung Nguyen, Chi Thang Duong, Thanh Tam Nguyen, Matthias Weidlich, Karl Aberer, Hongzhi Yin, and Xiaofang Zhou. 2017. Argument discovery via crowdsourcing. VLDB J. 26 (2017), 511–535.

[35]

Weike Pan, Evan Xiang, Nathan Liu, and Qiang Yang. 2010. Transfer learning in collaborative filtering for sparsity reduction. In Proceedings of the AAAI Conference on Artificial Intelligence. 230–235.

[36]

Weike Pan and Qiang Yang. 2013. Transfer learning in heterogeneous collaborative filtering domains. Artif. Intell. 197 (2013), 39–55.

Digital Library

[37]

Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 813–823.

Digital Library

[38]

Ruihong Qiu, Jingjing Li, Zi Huang, and Hongzhi Yin. 2019. Rethinking the item order in session-based recommendation with graph neural networks. In Proceedings of the International Conference on Information and Knowledge Management. 579–588.

[39]

Ruihong Qiu, Hongzhi Yin, Zi Huang, and Tong Chen. 2020. GAG: Global attributed graph neural network for streaming session-based recommendation. In Proceedings of the International Conference on Research and Development in Information Retrieval. 669–678.

[40]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 1 (2020), 5485–5551.

Digital Library

[41]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the Conference on Uncertainty in Artificial Intelligence. 452–461.

Digital Library

[42]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the ACM International World Wide Web Conferences. 811–820.

Digital Library

[43]

Shaghayegh Sahebi and Peter Brusilovsky. 2013. Cross-domain collaborative recommendation in a cold-start context: The impact of user profile size on the quality of recommendation. In Proceedings of the International Conference on User Modeling, Adaptation, and Personalization. 289–295.

[44]

Timo Schick and Hinrich Schütze. 2020. Exploiting cloze questions for few shot text classification and natural language inference. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics. 255–269.

[45]

Jin Shang, Mingxuan Sun, and Kevyn Collins-Thompson. 2018. Demographic inference via knowledge transfer in cross-domain recommender systems. In Proceedings of the IEEE International Conference on Data Mining. 1218–1223.

[46]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the ACM International Conference on Information and Knowledge Management. 1441–1450.

[47]

Wenchao Sun, Muyang Ma, Pengjie Ren, Yujie Lin, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2023. Parallel split-join networks for shared account cross-domain sequential recommendations. IEEE Transactions on Knowledge and Data Engineering 35, 4 (2023), 4106–4123.

[48]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 565–573.

Digital Library

[49]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017).

[50]

Cheng Wang, Mathias Niepert, and Hui Li. 2019. Recsys-dan: Discriminative adversarial networks for cross-domain recommender systems. IEEE Transactions on Neural Networks and Learning Systems 31, 8 (2019), 2731–2740.

[51]

Yuhao Wang, Xiangyu Zhao, Bo Chen, Qidong Liu, Huifeng Guo, Huanshuo Liu, Yichao Wang, Rui Zhang, and Ruiming Tang. 2023. PLATE: A prompt-enhanced paradigm for multi-scenario recommendations. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 1498–1507.

[52]

Yiqing Wu, Ruobing Xie, Yongchun Zhu, Fuzhen Zhuang, Ao Xiang, Xu Zhang, Leyu Lin, and Qing He. 2022. Selective fairness in recommendation via prompts. In Proceedings of the International Conference on Research and Development in Information Retrieval. 2657–2662.

[53]

Yiqing Wu, Ruobing Xie, Yongchun Zhu, Fuzhen Zhuang, Xu Zhang, Leyu Lin, and Qing He. 2022. Personalized prompts for sequential recommendation. arXiv:2205.09666 [cs.IR].

[54]

Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive learning for sequential recommendation. In Proceedings of the International Conference on Data Engineering. 1259–1273.

[55]

Xin Xin, Tiago Pimentel, Alexandros Karatzoglou, Pengjie Ren, Konstantina Christakopoulou, and Zhaochun Ren. 2022. Rethinking reinforcement learning for recommendation: A prompt perspective. In Proceedings of the International Conference on Research and Development in Information Retrieval. 1347–1357.

[56]

Hongzhi Yin, Bin Cui, Zi Huang, Weiqing Wang, Xian Wu, and Xiaofang Zhou. 2015. Joint modeling of users’ interests and mobility patterns for point-of-interest recommendation. In Proceedings of the ACM International Conference on Multimedia. 819–822.

[57]

Qian Zhang, Jie Lu, and Guangquan Zhang. 2020. Cross-domain recommendation with multiple sources. In Proceedings of the International Joint Conference on Neural Networks. 1–7.

[58]

Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the International Conference on Information and Knowledge Management. 1893–1902.

[59]

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. Int. J. Comput. Vis. 130, 9 (2022), 2337–2348.

Digital Library

[60]

Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. 2022. Filter-enhanced MLP is all you need for sequential recommendation. In Proceedings of the ACM International World Wide Web Conferences. 2388–2399.

[61]

Feng Zhu, Chaochao Chen, Yan Wang, Guanfeng Liu, and Xiaolin Zheng. 2019. Dtcdr: A framework for dual-target cross-domain recommendation. In Proceedings of the International Conference on Information and Knowledge Management. 1533–1542.

[62]

Feng Zhu, Yan Wang, Chaochao Chen, Guanfeng Liu, Mehmet A. Orgun, and Jia Wu. 2018. A deep framework for cross-domain and cross-system recommendations. In Proceedings of the International Joint Conference on Artificial Intelligence. 3711–3717.

[63]

Feng Zhu, Yan Wang, Chaochao Chen, Guanfeng Liu, and Xiaolin Zheng. 2020. A graphical and attentional framework for dual-target cross-domain recommendation. In Proceedings of the International Joint Conference on Artificial Intelligence. 3001–3008.

[64]

Feng Zhu, Yan Wang, Jun Zhou, Chaochao Chen, Longfei Li, and Guanfeng Liu. 2023. A unified framework for cross-domain and cross-system recommendations. IEEE Transactions on Knowledge and Data Engineering 35, 2 (2023), 1171–1184.

[65]

Yuehua Zhu, Bo Huang, Shaohua Jiang, Muli Yang, Yanhua Yang, and Wenliang Zhong. 2022. Progressive self-attention network with unsymmetrical positional encoding for sequential recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 2029–2033.

Cited By

Liu XLi BChen YLi XXu SYin H(2025)Disentangled Representations for Cross-Domain Recommendation via Heterogeneous Graph Contrastive LearningDatabase Systems for Advanced Applications10.1007/978-981-97-5555-4_3(35-50)Online publication date: 12-Jan-2025
https://doi.org/10.1007/978-981-97-5555-4_3
Wang XLu HXu LGuo LZhao X(2024)Attention-Based Difficulty Feature Enhancement for Knowledge Tracing2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650226(1-9)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650226

Index Terms

MCRPL: A Pretrain, Prompt, and Fine-tune Paradigm for Non-overlapping Many-to-one Cross-domain Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Contrastive Multi-view Interest Learning for Cross-domain Sequential Recommendation
Cross-domain recommendation (CDR), which leverages information collected from other domains, has been empirically demonstrated to effectively alleviate data sparsity and cold-start problems encountered in traditional recommendation systems. However, ...
DA-DAN: A Dual Adversarial Domain Adaption Network for Unsupervised Non-overlapping Cross-domain Recommendation
Unsupervised Non-overlapping Cross-domain Recommendation (UNCR) is the task that recommends source domain items to the target domain users, which is more challenging as the users are non-overlapped, and its learning process is unsupervised. Unsupervised ...
Multi-Domain Sequential Recommendation via Domain Space Learning
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

This paper explores Multi-Domain Sequential Recommendation (MDSR), an advancement of Multi-Domain Recommendation that incorporates sequential context. Recent MDSR approach exploits domain-specific sequences, decoupled from mixed-domain histories, to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems

ACM Transactions on Information Systems Volume 42, Issue 4

July 2024

751 pages

EISSN:1558-2868

DOI:10.1145/3613639

Editor:
Min Zhang
Tsinghua University, China

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 February 2024

Online AM: 22 January 2024

Accepted: 11 January 2024

Revised: 10 November 2023

Received: 14 August 2023

Published in TOIS Volume 42, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Natural Science Foundation of Shandong Province
CCF-Baidu Open Fund
Australian Research Council Future Fellowship
Humanities and Social Sciences Fund of the Ministry of Education

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
542
Total Downloads

Downloads (Last 12 months)542
Downloads (Last 6 weeks)38

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu XLi BChen YLi XXu SYin H(2025)Disentangled Representations for Cross-Domain Recommendation via Heterogeneous Graph Contrastive LearningDatabase Systems for Advanced Applications10.1007/978-981-97-5555-4_3(35-50)Online publication date: 12-Jan-2025
https://doi.org/10.1007/978-981-97-5555-4_3
Wang XLu HXu LGuo LZhao X(2024)Attention-Based Difficulty Feature Enhancement for Knowledge Tracing2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650226(1-9)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650226

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents