research-article

AdaFS: Adaptive Feature Selection in Deep Recommender System

Authors:

Xian WuAuthors Info & Claims

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 3309 - 3317

https://doi.org/10.1145/3534678.3539204

Published: 14 August 2022 Publication History

Abstract

Feature selection plays an impactful role in deep recommender systems, which selects a subset of the most predictive features, so as to boost the recommendation performance and accelerate model optimization. The majority of existing feature selection methods, however, aim to select only a fixed subset of features. This setting cannot fit the dynamic and complex environments of practical recommender systems, where the contribution of a specific feature varies significantly across user-item interactions. In this paper, we propose an adaptive feature selection framework, AdaFS, for deep recommender systems. To be specific, we develop a novel controller network to automatically select the most relevant features from the whole feature space, which fits the dynamic recommendation environment better. Besides, different from classic feature selection approaches, the proposed controller can adaptively score each example of user-item interactions, and identify the most informative features correspondingly for subsequent recommendation tasks. We conduct extensive experiments based on two public benchmark datasets from a real-world recommender system. Experimental results demonstrate the effectiveness of AdaFS, and its excellent transferability to the most popular deep recommendation models.

Supplemental Material

MP4 File

This is an introduction video for our proposed AdaFS (Adaptive feature selection) framework, which is an effective feature selection method to adaptively and dynamically select the most predictive features in the field of Deep Recommender System. We will introduce the background, motivation, framework, and experiment sections one by one in this video.

Download
19.57 MB

References

[1]

Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE TPAMI (2013).

Digital Library

[2]

Leo Breiman. 1997. Arcing the edge. Technical Report.

[3]

Bo Chen, Xiangyu Zhao, Yejing Wang, Wenqi Fan, Huifeng Guo, and Ruiming Tang. 2022. Automated Machine Learning for Deep Recommender Systems: A Survey. arXiv preprint arXiv:2204.01390 (2022).

[4]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proc. of DLRS.

Digital Library

[5]

Wenqi Fan, Tyler Derr, Xiangyu Zhao, Yao Ma, Hui Liu, Jianping Wang, Jiliang Tang, and Qing Li. 2021. Attacking Black-box Recommendations via Copying Cross-domain User Profiles. In Proc. of ICDE.

[6]

Wei Fan, Kunpeng Liu, Hao Liu, Ahmad Hariri, Dejing Dou, and Yanjie Fu. 2021. AutoGFS: Automated Group-based Feature Selection via Interactive Reinforcement Learning. In Proc. of SDM.

[7]

Wei Fan, Kunpeng Liu, Hao Liu, Pengyang Wang, Yong Ge, and Yanjie Fu. 2020. AutoFS: Automated Feature selection via diversity-aware interactive reinforcement learning. In Proc. of ICDM.

[8]

Seyed Mehdi Hazrati Fard, Ali Hamzeh, and Sattar Hashemi. 2013. Using reinforcement learning to find an optimal set of features. Comput. Math. with Appl. (2013).

[9]

Jerome H Friedman. 2017. The elements of statistical learning: Data mining, inference, and prediction. springer open.

[10]

Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, et al. 2021. Towards Long-term Fairness in Recommendation. In Proc. of WSDM.

Digital Library

[11]

Pablo M Granitto, Cesare Furlanello, Franco Biasioli, and Flavia Gasperi. 2006. Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products. Chemometr Intell Lab Syst (2006).

[12]

Huifeng Guo, Bo Chen, Ruiming Tang,Weinan Zhang, Zhenguo Li, and Xiuqiang He. 2021. An Embedding Learning Framework for Numerical Features in CTR Prediction. In Proc. of KDD.

Digital Library

[13]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proc. of IJCAI.

[14]

Isabelle Guyon and André Elisseeff. 2003. An introduction to variable and feature selection. Journal of machine learning research (2003).

[15]

Mark Andrew Hall et al. 1999. Correlation-based feature selection for machine learning. (1999).

[16]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proc. of ICML.

Digital Library

[17]

Yufan Jiang, Chi Hu, Tong Xiao, Chunliang Zhang, and Jingbo Zhu. 2019. Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition. In Proc. of EMNLP.

[18]

Wei Jin, Xiaorui Liu, Xiangyu Zhao, Yao Ma, Neil Shah, and Jiliang Tang. 2022. Automated Self-Supervised Learning for Graphs. In Proc. of ICLR.

[19]

Manas R Joglekar, Cong Li, Mei Chen, Taibai Xu, Xiaoming Wang, Jay K Adams, Pranav Khaitan, Jiahui Liu, and Quoc V Le. 2020. Neural input search for large scale recommendation models. In Proc. of KDD.

Digital Library

[20]

Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research (1996).

[21]

Farhan Khawar, Xu Hang, Ruiming Tang, Bin Liu, Zhenguo Li, and Xiuqiang He. 2020. AutoFeature: Searching for Feature Interactions and Their Architectures for Click-through Rate Prediction. In Proc. of CIKM.

Digital Library

[22]

Ron Kohavi and George H John. 1997. Wrappers for feature subset selection. Artificial intelligence (1997).

[23]

Mark Kroon and Shimon Whiteson. 2009. Automatic feature selection for modelbased reinforcement learning in factored MDPs. In Proc. of ICML.

[24]

Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proc. of KDD.

Digital Library

[25]

Huan Liu and Rudy Setiono. 1995. Chi2: Feature selection and discretization of numeric attributes. In Proc. of ICTAI.

[26]

Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. In Proc. of ICLR.

[27]

Haochen Liu, Xiangyu Zhao, Chong Wang, Xiaobing Liu, and Jiliang Tang. 2020. Automated Embedding Size Search in Deep Recommender Systems. In Proc. of SIGIR.

Digital Library

[28]

Kunpeng Liu, Yanjie Fu, Pengfei Wang, Le Wu, Rui Bo, and Xiaolin Li. 2019. Automating feature subspace exploration via multi-agent reinforcement learning. In Proc. of KDD.

Digital Library

[29]

Kunpeng Liu, Yanjie Fu, LeWu, Xiaolin Li, Charu Aggarwal, and Hui Xiong. 2021. Automated feature selection: A reinforcement learning perspective. IEEE TKDE (2021).

[30]

Siyi Liu, Chen Gao, Yihong Chen, Depeng Jin, and Yong Li. 2021. Learnable Embedding Sizes for Recommender Systems. In Proc. of ICLR.

[31]

Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen,Wenyuan Dai, and Qiang Yang. 2019. Autocross: Automatic feature crossing for tabular data in real-world applications. In Proc. of KDD.

Digital Library

[32]

Tom Mitchell. 1997. Machine learning. (1997).

[33]

Boaz Nadler and Ronald R Coifman. 2005. The prediction error in CLS and PLS: the importance of feature selection prior to multivariate calibration. Journal of Chemometrics: A Journal of the Chemometrics Society (2005).

[34]

Alexey Natekin and Alois Knoll. 2013. Gradient boosting machines, a tutorial. Frontiers in neurorobotics (2013).

[35]

Hanchuan Peng, Fuhui Long, and Chris Ding. 2005. Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy. IEEE TPAMI (2005).

[36]

Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameters Sharing. In Proc. of ICML.

[37]

Hassan Ramchoun, Mohammed Amine Janati Idrissi, Youssef Ghanou, and Mohamed Ettaouil. 2016. Multilayer Perceptron: Architecture Optimization and Training. Int. J. Interact. Multim. Artif. Intell. (2016).

[38]

Steffen Rendle. 2010. Factorization machines. In Proc. of ICDM.

Digital Library

[39]

J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative filtering recommender systems. In The adaptive web.

[40]

Bo Shu, Fuji Ren, and Yanwei Bao. 2018. Investigating Lstm with k-Max Pooling for Text Classification. In Proc. of ICICTA.

[41]

Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proc. of KDD.

Digital Library

[42]

Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved recurrent neural networks for session-based recommendations. In Proc. of DLRS.

Digital Library

[43]

Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological) (1996).

[44]

Michael Tsang, Dehua Cheng, Hanpeng Liu, Xue Feng, Eric Zhou, and Yan Liu. 2020. Feature Interaction Interpretability: A Case for Explaining Ad- Recommendation Systems via Neural Interaction Detection. In Proc. of ICLR.

[45]

YejingWang, Xiangyu Zhao, Tong Xu, and XianWu. 2022. AutoField: Automating Feature Selection in Deep Recommender Systems. In Proc. of WWW.

[46]

SvanteWold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems (1987).

[47]

Niannan Xue, Bin Liu, Huifeng Guo, Ruiming Tang, Fengwei Zhou, Stefanos P Zafeiriou, Yuzhou Zhang, Jun Wang, and Zhenguo Li. 2020. AutoHash: Learning Higher-order Feature Interactions for Deep CTR Prediction. IEEE TKDE (2020).

[48]

Yiming Yang and Jan O Pedersen. 1997. A comparative study on feature selection in text categorization. In Icml.

[49]

Lei Yu and Huan Liu. 2003. Feature selection for high-dimensional data: A fast correlation-based filter solution. In Proc. of ICML.

[50]

Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM CSUR (2019).

[51]

Weinan Zhang, Xiangyu Zhao, Li Zhao, Dawei Yin, Grace Hui Yang, and Alex Beutel. 2020. Deep Reinforcement Learning for Information Retrieval: Fundamentals and Advances. In Proc. of SIGIR.

Digital Library

[52]

Xiangyu Zhao. 2022. Adaptive and automated deep recommender systems. ACM SIGWEB Newsletter (2022).

[53]

Xiangyu Zhao, Haochen Liu, Wenqi Fan, Hui Liu, Jiliang Tang, and Chong Wang. 2021. AutoLoss: Automated Loss Function Search in Recommendations. In Proc. of KDD.

Digital Library

[54]

Xiangyu Zhao, Haochen Liu, Wenqi Fan, Hui Liu, Jiliang Tang, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, and Xiwang Yang. 2021. Autoemb: Automated embedding dimensionality search in streaming recommendations. In Proc. of ICDM.

[55]

Xiangyu Zhao, Haochen Liu, Hui Liu, Jiliang Tang, Weiwei Guo, Jun Shi, Sida Wang, Huiji Gao, and Bo Long. 2021. AutoDim: Field-aware Embedding Dimension Searchin Recommender Systems. In Proc. of WWW.

Digital Library

[56]

Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. 2019. Deep reinforcement learning for search, recommendation, and online advertising: a survey. ACM SIGWEB Newsletter (2019).

[57]

Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018. Deep Reinforcement Learning for Page-wise Recommendations. In Proc. of RecSys.

Digital Library

[58]

Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. 2018. Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning. In Proc. of KDD.

Digital Library

[59]

Barret Zoph and Quoc V. Le. 2017. Neural Architecture Search with Reinforcement Learning. arXiv:1611.01578 [cs.LG]

[60]

Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020. Neural Interactive Collaborative Filtering. In Proc. of SIGIR.

Digital Library

[61]

Lixin Zou, Long Xia, Linfang Hou, Xiangyu Zhao, and Dawei Yin. 2021. Data-Efficient Reinforcement Learning for Malaria Control. arXiv e-prints (2021).

Cited By

Kayange HMun JPark YChoi JChoi J(2024)A Hybrid Approach to Modeling Heart Rate Response for Personalized Fitness Recommendations Using Wearable DataElectronics10.3390/electronics1319388813:19(3888)Online publication date: 30-Sep-2024
https://doi.org/10.3390/electronics13193888
Zhang HZhao PMiao XShao YLiu ZYang TCui B(2024)Experimental Analysis of Large-Scale Learnable Vector Storage CompressionProceedings of the VLDB Endowment10.14778/3636218.363623417:4(808-822)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.14778/3636218.3636234
Zhang SWang MZhao XGuo RZhao YZhuang CGu JZhang ZYin H(2024)DNS-Rec: Data-aware Neural Architecture Search for Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688117(591-600)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688117
Show More Cited By

Index Terms

AdaFS: Adaptive Feature Selection in Deep Recommender System
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

AutoField: Automating Feature Selection in Deep Recommender Systems
WWW '22: Proceedings of the ACM Web Conference 2022

Feature quality has an impactful effect on recommendation performance. Thereby, feature selection is a critical process in developing deep learning-based recommender systems. Most existing deep recommender systems, however, focus on designing ...
Improving Accuracy of Recommender System by Item Clustering

Recommender System (RS) predicts user's ratings towards items, and then recommends highly-predicted items to user. In recent years, RS has been playing more and more important role in the agent research field. There have been a great deal of researches ...
A New Approach for Recommender System
ICACS '17: Proceedings of the 1st International Conference on Algorithms, Computing and Systems

In today's e-commerce environment, Collaborative Filtering (CF) is a widely used algorithm for recommender system, which is to identify the users who have similar preferences to the target user, and to predict the preference of the target user according ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2022

5033 pages

ISBN:9781450393850

DOI:10.1145/3534678

General Chairs:
Aidong Zhang
University of Virginia
,
Huzefa Rangwala
Amazon/George Mason University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

APRC - CityU New Research Initiatives
CCF-Tencent Open Fund
SIRG - CityU Strategic Interdisciplinary Research Grant

Conference

KDD '22

Sponsor:

KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2022

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
1,190
Total Downloads

Downloads (Last 12 months)313
Downloads (Last 6 weeks)37

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kayange HMun JPark YChoi JChoi J(2024)A Hybrid Approach to Modeling Heart Rate Response for Personalized Fitness Recommendations Using Wearable DataElectronics10.3390/electronics1319388813:19(3888)Online publication date: 30-Sep-2024
https://doi.org/10.3390/electronics13193888
Zhang HZhao PMiao XShao YLiu ZYang TCui B(2024)Experimental Analysis of Large-Scale Learnable Vector Storage CompressionProceedings of the VLDB Endowment10.14778/3636218.363623417:4(808-822)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.14778/3636218.3636234
Zhang SWang MZhao XGuo RZhao YZhuang CGu JZhang ZYin H(2024)DNS-Rec: Data-aware Neural Architecture Search for Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688117(591-600)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688117
Jia PWang YDu ZZhao XWang YChen BWang WGuo HTang RBaeza-Yates RBonchi F(2024)ERASE: Benchmarking Feature Selection Methods for Deep Recommender SystemsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671571(5194-5205)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671571
Chen BZhao XWang YFan WGuo HTang R(2024)A Comprehensive Survey on Automated Machine Learning for RecommendationsACM Transactions on Recommender Systems10.1145/36301042:2(1-38)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1145/3630104
Liu LWang WZhao XZhang ZZhang CLin SWang YZou LLiu ZWei XYin HLi QSerra ESpezzano F(2024)Efficient and Robust Regularized Federated RecommendationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679682(1452-1461)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679682
Liu DXian SWu YYang CTang XHe XMing ZHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)AutoDCS: Automated Decision Chain Selection in Deep Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657818(956-965)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657818
Liu DYang CTang XWang YLyu FLuo WHe XMing ZZhao XAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)MultiFS: Automated Multi-Scenario Feature Selection in Deep Recommender SystemsProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635859(434-442)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635859
Qu YChen TNguyen QYin HAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Budgeted Embedding Table For Recommender SystemsProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635778(557-566)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635778
Du ZChen JJia QWu CZhu JDong ZTang RChua TNgo CKumar RLauw HKa-Wei Lee R(2024)LightCS: Selecting Quadratic Feature Crosses in Linear ComplexityCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3648300(38-46)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3648300
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents