research-article

Reinforced PU-learning with Hybrid Negative Sampling Strategies for Recommendation

Authors:

Chiao-Ting Chen,

Chuan-Yun Sang,

Szu-Hao HuangAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology, Volume 14, Issue 3

Article No.: 57, Pages 1 - 25

https://doi.org/10.1145/3582562

Published: 08 May 2023 Publication History

Abstract

The data of recommendation systems typically only contain the purchased item as positive data and other un-purchased items as unlabeled data. To train a good recommendation model, in addition to the known positive information, we also need high-quality negative information. Capturing negative signals in positive and unlabeled data is challenging for recommendation systems. Most studies have used specific data and proposed negative sampling methods suitable to the data characteristics. Existing negative sampling strategies cannot automatically select suitable approaches for different data. However, this one-size-fits-all strategy often makes potential positive samples considered as negative, or truly negative samples considered as potential positive samples and recommend to users. In this way, it will not only turn down the recommendation result, but even also have an adverse effect. Accordingly, we propose a novel negative sampling model, Reinforced PU-learning with Hybrid Negative Sampling Strategies for Recommendation (RHNSR), which can combine multiple sampling strategies and dynamically adjust the proportions used by different sampling strategies. In addition, ensemble learning, which integrates various model sampling strategies for obtaining an improved solution, was applied to RHNSR. Extensive experiments were conducted on three real-world recommendation datasets, and the experimental results indicated that the proposed model significantly outperformed state-of-the-art baseline models and revealed significant improvements in precision and hit ratio (49.02% and 37.41%, respectively).

References

[1]

Rishabh Ahuja, Arun Solanki, and Anand Nayyar. 2019. Movie recommender system using K-means clustering and K-nearest neighbor. In Proceedings of the 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence). IEEE, 263–268.

[2]

Jessa Bekker and Jesse Davis. 2020. Learning from positive and unlabeled data: A survey. Mach. Learn. 109, 4 (2020), 719–760.

Digital Library

[3]

Gérard Biau and Erwan Scornet. 2016. A random forest guided tour. Test 25, 2 (2016), 197–227.

[4]

Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. 2013. Recommender systems survey. Knowl.-based Syst. 46 (2013), 109–132.

Digital Library

[5]

Hugo Caselles-Dupré, Florian Lesaint, and Jimena Royo-Letelier. 2018. Word2vec applied to recommendation: Hyperparameters matter. In Proceedings of the 12th ACM Conference on Recommender Systems. 352–356.

Digital Library

[6]

Chong Chen, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020. Jointly non-sampling learning for knowledge graph enhanced recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 189–198.

Digital Library

[7]

Ting Chen, Yizhou Sun, Yue Shi, and Liangjie Hong. 2017. On sampling strategies for neural network-based collaborative filtering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 767–776.

Digital Library

[8]

Yi-Ching Chou, Chiao-Ting Chen, and Szu-Hao Huang. 2022. Modeling behavior sequence for personalized fund recommendation with graphical deep collaborative filtering. Expert Syst. Applic. 192 (2022), 116311.

Digital Library

[9]

Marc Claesen, Frank De Smet, Johan A. K. Suykens, and Bart De Moor. 2015. A robust ensemble approach to learn from positive and unlabeled data using SVM base models. Neurocomputing 160 (2015), 73–84.

Digital Library

[10]

William W. Cohen. 1995. Fast effective rule induction. In Proceedings of the 12th International Conference on Machine Learning.

[11]

Jingtao Ding, Fuli Feng, Xiangnan He, Guanghui Yu, Yong Li, and Depeng Jin. 2018. An improved sampler for Bayesian personalized ranking by leveraging view data. In Proceedings of the the Web Conference. 13–14.

Digital Library

[12]

Charles Elkan and Keith Noto. 2008. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 213–220.

Digital Library

[13]

Saman Forouzandeh, Kamal Berahmand, and Mehrdad Rostami. 2021. Presentation of a recommender system with ensemble learning and graph embedding: A case on MovieLens. Multim. Tools Applic. 80, 5 (2021), 7805–7832.

Digital Library

[14]

Yoav Freund and Robert E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 1 (1997), 119–139.

Digital Library

[15]

Zhoutong Fu, Huiji Gao, Weiwei Guo, Sandeep Kumar Jha, Jun Jia, Xiaowei Liu, Bo Long, Jun Shi, Sida Wang, and Mingzhou Zhou. 2020. Deep learning for search and recommender systems in practice. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3515–3516.

Digital Library

[16]

Weiwei Guo, Huiji Gao, Jun Shi, Bo Long, Liang Zhang, Bee-Chung Chen, and Deepak Agarwal. 2019. Deep natural language processing for search and recommender systems. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3199–3200.

Digital Library

[17]

William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1025–1035.

Digital Library

[18]

Fengxiang He, Tongliang Liu, Geoffrey I. Webb, and Dacheng Tao. 2018. Instance-dependent PU learning by Bayesian optimal relabeling. arXiv preprint arXiv:1808.02180 (2018).

[19]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. 173–182.

Digital Library

[20]

Cho-Jui Hsieh, Nagarajan Natarajan, and Inderjit Dhillon. 2015. PU learning for matrix completion. In Proceedings of the International Conference on Machine Learning. PMLR, 2445–2453.

[21]

Pei-Ying Hsu, Chiao-Ting Chen, Chin Chou, and Szu-Hao Huang. 2022. Explainable mutual fund recommendation system developed based on knowledge graph embeddings. Appl. Intell. 52, 9 (2022), 10779–10804.

Digital Library

[22]

Tsan-Yin Hung and Szu-Hao Huang. 2022. Addressing the cold-start problem of recommendation systems for financial products by using few-shot deep learning. Appl. Intell. 52, 13 (2022), 15529–15546.

Digital Library

[23]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[24]

Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[25]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.

Digital Library

[26]

Jing Li, Feng Xia, Wei Wang, Zhen Chen, Nana Yaw Asabere, and Huizhen Jiang. 2014. Acrec: A co-authorship based random walk model for academic collaboration recommendation. In Proceedings of the 23rd International Conference on World Wide Web. 1209–1214.

[27]

Babak Loni, Roberto Pagano, Martha Larson, and Alan Hanjalic. 2016. Bayesian personalized ranking with multi-channel user feedback. In Proceedings of the 10th ACM Conference on Recommender Systems. 361–364.

Digital Library

[28]

Fantine Mordelet and J.-P. Vert. 2014. A bagging SVM to learn from positive and unlabeled examples. Pattern Recog. Lett. 37 (2014), 201–209.

Digital Library

[29]

Fedelucio Narducci, Pierpaolo Basile, Marco de Gemmis, Pasquale Lops, and Giovanni Semeraro. 2020. An investigation on the user interaction modes of conversational recommender systems for the music domain. User Model. User-adapt. Interact. 30, 2 (2020), 251–284.

Digital Library

[30]

Alexey Natekin and Alois Knoll. 2013. Gradient boosting machines, a tutorial. Front. Neurorobot. 7 (2013), 21.

[31]

Thi Thanh Sang Nguyen. 2019. Model-based book recommender systems using naïve bayes enhanced with optimal feature selection. In Proceedings of the 8th International Conference on Software and Computer Applications. 217–222.

Digital Library

[32]

Curtis G. Northcutt, Tailin Wu, and Isaac L. Chuang. 2017. Learning with confident examples: Rank pruning for robust classification with noisy labels. arXiv preprint arXiv:1705.01936 (2017).

[33]

Michael J. Pazzani. 1999. A framework for collaborative, content-based and demographic filtering. Artif. Intell. Rev. 13, 5 (1999), 393–408.

Digital Library

[34]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).

[35]

Omer Sagi and Lior Rokach. 2018. Ensemble learning: A survey. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 8, 4 (2018), e1249.

[36]

Xiaoyuan Su and Taghi M. Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Adv. Artif. Intell. 2009 (2009), 4.

Digital Library

[37]

Panagiotis Symeonidis, Alexandros Nanopoulos, and Yannis Manolopoulos. 2009. MoviExplain: A recommender system with explanations. In Proceedings of the 3rd ACM Conference on Recommender Systems. 317–320.

Digital Library

[38]

Virginia Tsintzou, Evaggelia Pitoura, and Panayiotis Tsaparas. 2018. Bias disparity in recommendation systems. arXiv preprint arXiv:1811.01461 (2018).

[39]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[40]

Guangtao Wang, Qinbao Song, and Xiaoyan Zhu. 2021. Ensemble learning based classification algorithm recommendation. arXiv preprint arXiv:2101.05993 (2021).

[41]

Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017. IRGAN: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 515–524.

Digital Library

[42]

Ting-Yun Wang, Chiao-Ting Chen, Ju-Chun Huang, and Szu-Hao Huang. 2023. Modeling cross-session information with multi-interest graph neural networks for the next-item recommendation. ACM Trans. Knowl. Discov. Data 17, 1 (2023), 1–28.

Digital Library

[43]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 950–958.

Digital Library

[44]

Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, and Tat-Seng Chua. 2020. Reinforced negative sampling over knowledge graph for recommendation. In Proceedings of the Web Conference. 99–109.

Digital Library

[45]

Shiwen Wu, Fei Sun, Wentao Zhang, and Bin Cui. 2020. Graph neural networks in recommender systems: A survey. arXiv preprint arXiv:2011.02260 (2020).

[46]

Xu Yu, Qinglong Peng, Lingwei Xu, Feng Jiang, Junwei Du, and Dunwei Gong. 2021. A selective ensemble learning based two-sided cross-domain collaborative filtering algorithm. Inf. Process. Manag. 58, 6 (2021), 102691.

Digital Library

[47]

Dell Zhang and Wee Sun Lee. 2005. A simple probabilistic approach to learning from positive and unlabeled examples. In Proceedings of the 5th Annual UK Workshop on Computational Intelligence (UKCI). 83–87.

[48]

Weinan Zhang, Tianqi Chen, Jun Wang, and Yong Yu. 2013. Optimizing top-n collaborative filtering via dynamic negative item sampling. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. 785–788.

Digital Library

Cited By

Chen CLee CHuang SPeng W(2024)Credit Card Fraud Detection via Intelligent Sampling and Self-supervised LearningACM Transactions on Intelligent Systems and Technology10.1145/364128315:2(1-29)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3641283
Liu JChen CLee CHuang S(2024)Evolving Knowledge Graph Representation Learning with Multiple Attention Strategies for Citation Recommendation SystemACM Transactions on Intelligent Systems and Technology10.1145/363527315:2(1-26)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3635273
Hsu CChen CHuang S(2023)Adaptive Adversarial Contrastive Learning for Cross-Domain RecommendationACM Transactions on Knowledge Discovery from Data10.1145/363025918:3(1-34)Online publication date: 9-Dec-2023
https://dl.acm.org/doi/10.1145/3630259
Show More Cited By

Index Terms

Reinforced PU-learning with Hybrid Negative Sampling Strategies for Recommendation
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
    2. Machine learning approaches
      1. Learning latent representations
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Revisiting Negative Sampling vs. Non-sampling in Implicit Recommendation
Recommendation systems play an important role in alleviating the information overload issue. Generally, a recommendation model is trained to discern between positive (liked) and negative (disliked) instances for each user. However, under the open-world ...
Reinforced Negative Sampling over Knowledge Graph for Recommendation
WWW '20: Proceedings of The Web Conference 2020

Properly handling missing data is a fundamental challenge in recommendation. Most present works perform negative sampling from unobserved data to supply the training of recommender models with negative signals. Nevertheless, existing negative sampling ...
False Negative Sample Aware Negative Sampling for Recommendation
Advances in Knowledge Discovery and Data Mining
Abstract
Negative sampling plays a key role in implicit feedback collaborative filtering. It draws high-quality negative samples from a large number of uninteracted samples. Existing methods primarily focus on hard negative samples, while overlooking the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 14, Issue 3

June 2023

451 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3587032

Editor:
Huan Liu
Arizona State University, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2023

Online AM: 06 February 2023

Accepted: 09 January 2023

Revised: 25 November 2022

Received: 22 December 2021

Published in TIST Volume 14, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Ministry of Science and Technology, Taiwan
Financial Technology (FinTech) Innovation Research Center, National Yang Ming Chiao Tung University

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
395
Total Downloads

Downloads (Last 12 months)258
Downloads (Last 6 weeks)29

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen CLee CHuang SPeng W(2024)Credit Card Fraud Detection via Intelligent Sampling and Self-supervised LearningACM Transactions on Intelligent Systems and Technology10.1145/364128315:2(1-29)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3641283
Liu JChen CLee CHuang S(2024)Evolving Knowledge Graph Representation Learning with Multiple Attention Strategies for Citation Recommendation SystemACM Transactions on Intelligent Systems and Technology10.1145/363527315:2(1-26)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3635273
Hsu CChen CHuang S(2023)Adaptive Adversarial Contrastive Learning for Cross-Domain RecommendationACM Transactions on Knowledge Discovery from Data10.1145/363025918:3(1-34)Online publication date: 9-Dec-2023
https://dl.acm.org/doi/10.1145/3630259
Liu ZLi HChen GOuyang YRong WXiong ZFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)PopDCL: Popularity-aware Debiased Contrastive Loss for Collaborative FilteringProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615009(1482-1492)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615009
Yang YChen CLu THuang S(2023)Hierarchical Reinforcement Learning for Conversational Recommendation With Knowledge Graph Reasoning and Heterogeneous QuestionsIEEE Transactions on Services Computing10.1109/TSC.2023.326939616:5(3439-3452)Online publication date: Sep-2023
https://doi.org/10.1109/TSC.2023.3269396

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents