research-article

ColdNAS: Search to Modulate for User Cold-Start Recommendation

Authors:

Quanming YaoAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 1021 - 1031

https://doi.org/10.1145/3543507.3583344

Published: 30 April 2023 Publication History

Abstract

Making personalized recommendation for cold-start users, who only have a few interaction histories, is a challenging problem in recommendation systems. Recent works leverage hypernetworks to directly map user interaction histories to user-specific parameters, which are then used to modulate predictor by feature-wise linear modulation function. These works obtain the state-of-the-art performance. However, the physical meaning of scaling and shifting in recommendation data is unclear. Instead of using a fixed modulation function and deciding modulation position by expertise, we propose a modulation framework called ColdNAS for user cold-start problem, where we look for proper modulation structure, including function and position, via neural architecture search. We design a search space which covers broad models and theoretically prove that this search space can be transformed to a much smaller space, enabling an efficient and robust one-shot search algorithm. Extensive experimental results on benchmark datasets show that ColdNAS consistently performs the best. We observe that different modulation functions lead to the best performance on different datasets, which validates the necessity of designing a searching-based method. Codes are available at https://github.com/LARS-research/ColdNAS.

References

[1]

Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. 2017. Designing neural network architectures using reinforcement learning. In International Conference on Learning Representations.

[2]

James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, 2 (2012).

[3]

Homanga Bharadhwaj. 2019. Meta-learning for user cold-start recommendation. In International Joint Conference on Neural Networks. 1–8.

[4]

Marc Brockschmidt. 2020. GNN-FiLM: Graph neural networks with feature-wise linear modulation. In International Conference on Machine Learning. 1144–1152.

[5]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, 2016. Wide & deep learning for recommender systems. In Workshop on Deep Learning for Recommender Systems. 7–10.

[6]

Manqing Dong, Feng Yuan, Lina Yao, Xiwei Xu, and Liming Zhu. 2020. MAMO: Memory-augmented meta-optimization for cold-start recommendation. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 688–697.

[7]

Xidong Feng, Chen Chen, Dong Li, Mengchen Zhao, Jianye Hao, and Jun Wang. 2021. CMML: Contextual modulation meta learning for cold-start recommendation. In ACM International Conference on Information and Knowledge Management. 484–493.

[8]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning. 1126–1135.

[9]

Chen Gao, Yinfeng Li, Quanming Yao, Depeng Jin, and Yong Li. 2021. Progressive feature interaction search for deep sparse network. In Advances in Neural Information Processing Systems, Vol. 34. 392–403.

[10]

Chen Gao, Quanming Yao, Depeng Jin, and Yong Li. 2021. Efficient Data-specific Model Search for Collaborative Filtering. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 415–425.

[11]

David Ha, Andrew Dai, and Quoc V Le. 2017. Hypernetworks. In International Conference on Learning Representations.

[12]

F Maxwell Harper and Joseph A Konstan. 2015. The MovieLens datasets: History and context. ACM Transactions on Interactive Intelligent Systems 5, 4 (2015), 1–19.

[13]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In International Conference on World Wide Web. 173–182.

[14]

Kurt Hornik, Maxwell Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. Neural Networks 2, 5 (1989), 359–366.

[15]

Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017. Collaborative metric learning. In International Conference on World Wide Web. 193–201.

[16]

Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. 2019. Automated machine learning: methods, systems, challenges. Springer Nature.

[17]

Haifeng Jin, Qingquan Song, and Xia Hu. 2019. Auto-Keras: An efficient neural architecture search system. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1946–1956.

[18]

Yehuda Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 426–434.

[19]

Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019. MeLU: Meta-learned user preference estimator for cold-start recommendation. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1073–1082.

[20]

Jovian Lin, Kazunari Sugiyama, Min-Yen Kan, and Tat-Seng Chua. 2013. Addressing cold-start in app recommendation: latent user models constructed from twitter followers. In International ACM SIGIR Conference on Research and Development in Information Retrieval. 283–292.

[21]

Xixun Lin, Jia Wu, Chuan Zhou, Shirui Pan, Yanan Cao, and Bin Wang. 2021. Task-adaptive neural process for user cold-start recommendation. In The Web Conference. 1306–1316.

[22]

Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020. AutoFIS: Automatic feature interaction selection in factorization models for click-through rate prediction. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2636–2645.

[23]

Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable architecture search. In International Conference on Learning Representations.

[24]

Yuanfu Lu, Yuan Fang, and Chuan Shi. 2020. Meta-learning on heterogeneous information networks for cold-start recommendation. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1563–1573.

[25]

Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu. 2018. Neural architecture optimization. In Advances in Neural Information Processing Systems, Vol. 31. 7827–7838.

[26]

Haoyu Pang, Fausto Giunchiglia, Ximing Li, Renchu Guan, and Xiaoyue Feng. 2022. PNMTA: A pretrained network modulation and task adaptation approach for user cold-start recommendation. In The Web Conference. 348–359.

[27]

Yoon-Joo Park and Alexander Tuzhilin. 2008. The long tail of recommender systems and how to leverage it. In ACM Conference on Recommender Systems. 11–18.

[28]

Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. 2018. FiLM: Visual reasoning with a general conditioning layer. In AAAI Conference on Artificial Intelligence, Vol. 32. 3942–3951.

[29]

Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning. 4095–4104.

[30]

Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. 2019. Regularized evolution for image classifier architecture search. In AAAI Conference on Artificial Intelligence, Vol. 33. 4780–4789.

[31]

James Requeima, Jonathan Gordon, John Bronskill, Sebastian Nowozin, and Richard E Turner. 2019. Fast and flexible multi-task classification using conditional neural adaptive processes. In Advances in Neural Information Processing Systems, Vol. 32. 7957–7968.

[32]

Paul Resnick and Hal R Varian. 1997. Recommender systems. Commun. ACM 40, 3 (1997), 56–58.

[33]

Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. 2002. Methods and metrics for cold-start recommendations. In International ACM SIGIR Conference on Research and Development in Information Retrieval. 253–260.

[34]

Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015. AutoRec: Autoencoders meet collaborative filtering. In International Conference on World Wide Web. 111–112.

[35]

Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 945–955.

[36]

Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. 2017. DropoutNet: Addressing cold start in recommender systems. In Advances in Neural Information Processing Systems, Vol. 30. 4957–4966.

[37]

Li Wang, Binbin Jin, Zhenya Huang, Hongke Zhao, Defu Lian, Qi Liu, and Enhong Chen. 2021. Preference-adaptive meta-learning for cold-start recommendation. In International Joint Conference on Artificial Intelligence. 1607–1614.

[38]

Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. 2020. Generalizing from a few examples: A survey on few-shot learning. Comput. Surveys 53, 3 (2020), 1–34.

[39]

Yuexiang Xie, Zhen Wang, Yaliang Li, Bolin Ding, Nezihe Merve Gürel, Ce Zhang, Minlie Huang, Wei Lin, and Jingren Zhou. 2021. FIVES: Feature interaction via edge search for large-scale tabular data. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3795–3805.

[40]

Yibo Yang, Hongyang Li, Shan You, Fei Wang, Chen Qian, and Zhouchen Lin. 2020. ISTA-NAS: Efficient and consistent neural architecture search by sparse coding. In Cross attention network for few-shot classification, Vol. 33. 10503–10513.

[41]

Quanming Yao, Xiangning Chen, James T Kwok, Yong Li, and Cho-Jui Hsieh. 2020. Efficient neural interaction function search for collaborative filtering. In The Web Conference. 1660–1670.

[42]

Quanming Yao, Ju Xu, Wei-Wei Tu, and Zhanxing Zhu. 2020. Efficient neural architecture search via proximal iterations. In AAAI Conference on Artificial Intelligence, Vol. 34. 6664–6671.

[43]

Kaicheng Yu, Christian Sciuto, Martin Jaggi, Claudiu Musat, and Mathieu Salzmann. 2019. Evaluating the search phase of neural architecture search. In International Conference on Learning Representations.

[44]

Runsheng Yu, Yu Gong, Xu He, Bo An, Yu Zhu, Qingwen Liu, and Wenwu Ou. 2021. Personalized adaptive meta learning for cold-start user preference prediction. In AAAI Conference on Artificial Intelligence, Vol. 35. 10772–10780.

[45]

Yongqi Zhang, Quanming Yao, and James T Kwok. 2023. Bilinear scoring function search for knowledge graph learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 2 (2023), 1458–1473.

[46]

Xiangyu Zhao, Haochen Liu, Wenqi Fan, Hui Liu, Jiliang Tang, and Chong Wang. 2021. AutoLoss: Automated loss function search in recommendations. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3959–3967.

[47]

Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. In International Conference on World Wide Web. 22–32.

Cited By

Xu XDong HXiang HHu XLi XXia XZhang XQi LDou W(2025)C2lRec: Causal Contrastive Learning for User Cold-start Recommendation with Social VariableACM Transactions on Information Systems10.1145/3711858Online publication date: 9-Jan-2025
https://doi.org/10.1145/3711858
Wang YPiao HDong DYao QZhou JBaeza-Yates RBonchi F(2024)Warming Up Cold-Start CTR Prediction by Learning Item-Specific Feature InteractionsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671784(3233-3244)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671784
Zhang ZLiu QHu ZZhan YHuang ZGao WMao QChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Enhancing Fairness in Meta-learned User Modeling via Adaptive SamplingProceedings of the ACM Web Conference 202410.1145/3589334.3645369(3241-3252)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645369
Show More Cited By

Index Terms

ColdNAS: Search to Modulate for User Cold-Start Recommendation
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Pairwise preference regression for cold-start recommendation
RecSys '09: Proceedings of the third ACM conference on Recommender systems

Recommender systems are widely used in online e-commerce applications to improve user engagement and then to increase revenue. A key challenge for recommender systems is providing high quality recommendation to users in ``cold-start" situations. We ...
Cold-Start Item and User Recommendation with Decoupled Completion and Transduction
RecSys '15: Proceedings of the 9th ACM Conference on Recommender Systems

A major challenge in collaborative filtering based recommender systems is how to provide recommendations when rating data is sparse or entirely missing for a subset of users or items, commonly known as the cold-start problem. In recent years, there has ...
Cold-Start Recommendation with Provable Guarantees: A Decoupled Approach

Although the matrix completion paradigm provides an appealing solution to the collaborative filtering problem in recommendation systems, some major issues, such as data sparsity and cold-start problems, still remain open. In particular, when the rating ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Available / v1.1

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Natural Science Foundation of China
CCF- Baidu Open Fund

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
385
Total Downloads

Downloads (Last 12 months)120
Downloads (Last 6 weeks)4

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu XDong HXiang HHu XLi XXia XZhang XQi LDou W(2025)C2lRec: Causal Contrastive Learning for User Cold-start Recommendation with Social VariableACM Transactions on Information Systems10.1145/3711858Online publication date: 9-Jan-2025
https://doi.org/10.1145/3711858
Wang YPiao HDong DYao QZhou JBaeza-Yates RBonchi F(2024)Warming Up Cold-Start CTR Prediction by Learning Item-Specific Feature InteractionsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671784(3233-3244)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671784
Zhang ZLiu QHu ZZhan YHuang ZGao WMao QChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Enhancing Fairness in Meta-learned User Modeling via Adaptive SamplingProceedings of the ACM Web Conference 202410.1145/3589334.3645369(3241-3252)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645369
Dong DWu SWang YZhou JWang H(2024)ColdU: User Cold-start Recommendation with User-specific Modulation2024 IEEE Conference on Artificial Intelligence (CAI)10.1109/CAI59869.2024.00069(326-331)Online publication date: 25-Jun-2024
https://doi.org/10.1109/CAI59869.2024.00069
Wang CWang MWang XTan Y(2024)IPSRM: An intent perceived sequential recommendation modelJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2024.10220636:9(102206)Online publication date: Nov-2024
https://doi.org/10.1016/j.jksuci.2024.102206
Cai RLu RChen WHao Z(2024)Counterfactual contextual bandit for recommendation under delayed feedbackNeural Computing and Applications10.1007/s00521-024-09800-036:23(14599-14613)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00521-024-09800-0

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten