research-article

Enhancing CTR Prediction with Context-Aware Feature Representation Learning

Authors:

Ning GuAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 343 - 352

https://doi.org/10.1145/3477495.3531970

Published: 07 July 2022 Publication History

Abstract

CTR prediction has been widely used in the real world. Many methods model feature interaction to improve their performance. However, most methods only learn a fixed representation for each feature without considering the varying importance of each feature under different contexts, resulting in inferior performance. Recently, several methods tried to learn vector-level weights for feature representations to address the fixed representation issue. However, they only produce linear transformations to refine the fixed feature representations, which are still not flexible enough to capture the varying importance of each feature under different contexts. In this paper, we propose a novel module named Feature Refinement Network (FRNet), which learns context-aware feature representations at bit-level for each feature in different contexts. FRNet consists of two key components: 1) Information Extraction Unit (IEU), which captures contextual information and cross-feature relationships to guide context-aware feature refinement; and 2) Complementary Selection Gate (CSGate), which adaptively integrates the original and complementary feature representations learned in IEU with bit-level weights. Notably, FRNet is orthogonal to existing CTR methods and thus can be applied in many existing methods to boost their performance. Comprehensive experiments are conducted to verify the effectiveness, efficiency, and compatibility of FRNet.

Supplementary Material

MP4 File (SIGIR22-fp0260.mp4)

The Presentation video of fp0260 - ''Enhancing CTR Prediction with Context-Aware Feature Representation Learning.''

Download
15.40 MB

References

[1]

Bo Chen, Yichao Wang, Zhirong Liu, Ruiming Tang, Wei Guo, Hongkun Zheng, Weiwei Yao, Muyu Zhang, and Xiuqiang He. 2021. Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3757--3766.

Digital Library

[2]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7--10.

Digital Library

[3]

Weiyu Cheng, Yanyan Shen, and Linpeng Huang. 2020. Adaptive factorization network: Learning adaptive-order feature interactions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 3609--3616.

[4]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191--198.

Digital Library

[5]

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In International Conference on Machine Learning. PMLR, 1243--1252.

[6]

Huifeng Guo, Bo Chen, Ruiming Tang, Weinan Zhang, Zhenguo Li, and Xiuqiang He. 2021. An embedding learning framework for numerical features in ctr prediction. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2910--2918.

Digital Library

[7]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1725--1731.

[8]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026--1034.

Digital Library

[9]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[10]

Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 355--364.

Digital Library

[11]

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. 2014. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising. 1--9.

Digital Library

[12]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.

[13]

Tongwen Huang, Qingyun She, Zhiqiang Wang, and Junlin Zhang. 2020. GateNet: Gating-Enhanced Deep Network for Click-Through Rate Prediction. arXiv preprint arXiv:2007.03519 (2020).

[14]

Tongwen Huang, Zhiqi Zhang, and Junlin Zhang. 2019. FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction. In Proceedings of the 13th ACM Conference on Recommender Systems. 169--177.

Digital Library

[15]

Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware factorization machines for CTR prediction. In Proceedings of the 10th ACM Conference on Recommender Systems. 43--50.

Digital Library

[16]

Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR (Poster).

[17]

Zeyu Li, Wei Cheng, Yang Chen, Haifeng Chen, and Wei Wang. 2020 a. Interpretable click-through rate prediction through hierarchical attention. In Proceedings of the 13th International Conference on Web Search and Data Mining. 313--321.

Digital Library

[18]

Z Li, J Zhang, Y Gong, Y Yao, and Q Wu. 2020 b. Field-wise Learning for Multi-field Categorical Data. In Conference on Neural Information Processing Systems.

[19]

Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1754--1763.

Digital Library

[20]

Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019. Feature generation by convolutional neural network for click-through rate prediction. In The World Wide Web Conference. 1119--1129.

Digital Library

[21]

Wantong Lu, Yantao Yu, Yongzhe Chang, Zhen Wang, Chenhui Li, and Bo Yuan. 2020. A Dual Input-aware Factorization Machine for CTR Prediction. In IJCAI. 3139--3145.

[22]

Yuanfei Luo, Hao Zhou, Wei-Wei Tu, Yuqiang Chen, Wenyuan Dai, and Qiang Yang. 2020. Network On Network for Tabular Data Classification in Real-world Applications. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2317--2326.

Digital Library

[23]

Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-weighted factorization machines for click-through rate prediction in display advertising. In Proceedings of the 2018 World Wide Web Conference. 1349--1357.

Digital Library

[24]

Yujie Pan, Jiangchao Yao, Bo Han, Kunyang Jia, Ya Zhang, and Hongxia Yang. 2021. Click-through Rate Prediction with Auto-Quantized Contrastive Learning. arXiv preprint arXiv:2109.13921 (2021).

[25]

Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-based neural networks for user response prediction over multi-field categorical data. ACM Transactions on Information Systems (TOIS), Vol. 37, 1 (2018), 1--35.

Digital Library

[26]

Steffen Rendle. 2012. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 3, 3 (2012), 1--22.

Digital Library

[27]

Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web. 521--530.

Digital Library

[28]

Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. Autoint: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1161--1170.

Digital Library

[29]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, 11 (2008).

[30]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[31]

Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In Proceedings of the ADKDD'17. 1--7.

Digital Library

[32]

Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. 2021 b. DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems. In Proceedings of the Web Conference 2021. 1785--1797.

Digital Library

[33]

Zhiqiang Wang, Qingyun She, and Junlin Zhang. 2021 a. MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask. arXiv preprint arXiv:2102.07619 (2021).

[34]

Zhikun Wei, Xin Wang, and Wenwu Zhu. 2021. AutoIAS: Automatic Integrated Architecture Searcher for Click-Trough Rate Prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2101--2110.

Digital Library

[35]

Shu Wu, Feng Yu, Xueli Yu, Qiang Liu, Liang Wang, Tieniu Tan, Jie Shao, and Fan Huang. 2020. TFNet: Multi-Semantic Feature Interaction for CTR Prediction. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1885--1888.

Digital Library

[36]

Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: learning the weight of feature interactions via attention networks. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 3119--3125.

[37]

Yi Yang, Baile Xu, Shaofeng Shen, Furao Shen, and Jian Zhao. 2020. Operation-aware Neural Networks for user response prediction. Neural Networks, Vol. 121 (2020), 161--168.

Digital Library

[38]

Runlong Yu, Yuyang Ye, Qi Liu, Zihan Wang, Chunfeng Yang, Yucheng Hu, and Enhong Chen. 2021. XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction. In PAKDD (2). Springer, 436--447.

[39]

Yantao Yu, Zhen Wang, and Bo Yuan. 2019. An Input-aware Factorization Machine for Sparse Prediction. In IJCAI. 1466--1472.

[40]

Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. In European conference on information retrieval. Springer, 45--57.

[41]

Keke Zhao, Xing Zhao, Qi Cao, and Linjian Mo. 2021 b. A Non-sequential Approach to Deep User Interest Model for Click-Through Rate Prediction. arXiv preprint arXiv:2104.06312 (2021).

[42]

Zihao Zhao, Zhiwei Fang, Yong Li, Changping Peng, Yongjun Bao, and Weipeng Yan. 2020. Dimension Relation Modeling for Click-Through Rate Prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2333--2336.

Digital Library

[43]

Zhishan Zhao, Sen Yang, Guohui Liu, Dawei Feng, and Kele Xu. 2021 a. FINT: Field-aware INTeraction Neural Network For CTR Prediction. arXiv preprint arXiv:2107.01999 (2021).

[44]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059--1068.

Digital Library

Cited By

Guan ZYang JYang YZhu HLi WXiong H(2024)JobFormer: Skill-Aware Job Recommendation with Semantic-Enhanced TransformerACM Transactions on Knowledge Discovery from Data10.1145/370173519:1(1-20)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701735
Li HSang LZhang YZhang YCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)SimCEN: Simple Contrast-enhanced Network for CTR PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681203(2311-2320)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681203
Fan ZLi FLiu HHe JDu XGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)PoseRec: 3D Human Pose Driven Online Advertisement Recommendation for Micro-videosProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658048(37-45)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658048
Show More Cited By

Index Terms

Enhancing CTR Prediction with Context-Aware Feature Representation Learning
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

CL4CTR: A Contrastive Learning Framework for CTR Prediction
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining

Many Click-Through Rate (CTR) prediction works focused on designing advanced architectures to model complex feature interactions but neglected the importance of feature representation learning, e.g., adopting a plain embedding layer for each feature, ...
Feature Representation Enhancing by Context Sensitive Information in CTR Prediction
Advanced Data Mining and Applications
Abstract
Click-Through-Rate (CTR) is a fundamental metric used to assess the efficacy of recommendation systems. In the past, most CTR prediction approaches focused on modeling the cross feature of various feature fields to improve the accuracy of CTR ...
Optimizing Feature Set for Click-Through Rate Prediction
WWW '23: Proceedings of the ACM Web Conference 2023

Click-through prediction (CTR) models transform features into latent vectors and enumerate possible feature interactions to improve performance based on the input feature set. Therefore, when selecting an optimal feature set, we should consider the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the National Natural Science Foundation of China (NSFC)

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
502
Total Downloads

Downloads (Last 12 months)124
Downloads (Last 6 weeks)13

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Guan ZYang JYang YZhu HLi WXiong H(2024)JobFormer: Skill-Aware Job Recommendation with Semantic-Enhanced TransformerACM Transactions on Knowledge Discovery from Data10.1145/370173519:1(1-20)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701735
Li HSang LZhang YZhang YCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)SimCEN: Simple Contrast-enhanced Network for CTR PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681203(2311-2320)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681203
Fan ZLi FLiu HHe JDu XGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)PoseRec: 3D Human Pose Driven Online Advertisement Recommendation for Micro-videosProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658048(37-45)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658048
Tian ZShi YWu XZhao WWen JBaeza-Yates RBonchi F(2024)Rotative Factorization MachinesProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671740(2912-2923)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671740
Huang JCai GZhu JDong ZTang RZhang WYu YChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Recall-Augmented Ranking: Enhancing Click-Through Rate Prediction Accuracy with Cross-Stage DataCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651551(830-833)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651551
Chen XCheng ZYao JJu CHuang WLan JZeng XXiao SChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Enhancing Cross-Domain Click-Through Rate Prediction via Explicit Feature AugmentationCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3648341(423-432)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3648341
Huang YWang SGao MWei XLi CLuo CZhu YXiao XLuo YChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Entire Chain Uplift Modeling with Context-Enhanced Learning for Intelligent MarketingCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3648320(226-234)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3648320
Zhang YChen MChen RZhao CYuan MSun Z(2024)Bayesian attention‐based user behaviour modelling for click‐through rate predictionCAAI Transactions on Intelligence Technology10.1049/cit2.123439:5(1320-1330)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1049/cit2.12343
Ye JLiu SQiu CZhang Z(2024)Learning multi-granularity semantic interactive representation for joint low-light image enhancement and super-resolutionInformation Fusion10.1016/j.inffus.2024.102467110(102467)Online publication date: Oct-2024
https://doi.org/10.1016/j.inffus.2024.102467
Li FYang ZGui Y(2024)SES-yolov5: small object graphics detection and visualization applicationsThe Visual Computer10.1007/s00371-024-03591-0Online publication date: 3-Aug-2024
https://doi.org/10.1007/s00371-024-03591-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents