research-article

Improving Ad Click Prediction by Considering Non-displayed Events

Authors:

Meng-Yuan Yang,

Chih-Yao Chang,

Chih-Jen LinAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 329 - 338

https://doi.org/10.1145/3357384.3358058

Published: 03 November 2019 Publication History

Abstract

Click-through rate (CTR) prediction is the core problem of building advertising systems. Most existing state-of-the-art approaches model CTR prediction as binary classification problems, where displayed events with and without click feedbacks are respectively considered as positive and negative instances for training and offline validation. However, due to the selection mechanism applied in most advertising systems, a selection bias exists between distributions of displayed and non-displayed events. Conventional CTR models ignoring the bias may have inaccurate predictions and cause a loss of the revenue. To alleviate the bias, we need to conduct counterfactual learning by considering not only displayed events but also non-displayed events. In this paper, through a review of existing approaches of counterfactual learning, we point out some difficulties for applying these approaches for CTR prediction in a real-world advertising system. To overcome these difficulties, we propose a novel framework for counterfactual CTR prediction. In experiments, we compare our proposed framework against state-of-the-art conventional CTR models and existing counterfactual learning approaches. Experimental results show significant improvements.

References

[1]

Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. 2017. A generic coordinate descent framework for learning from implicit feedback. In WWW .

[2]

Stephen Bonner and Flavian Vasile. 2018. Causal Embeddings for Recommendation. In RecSys .

[3]

Olivier Chapelle, Eren Manavoglu, and Romer Rosales. 2015. Simple and scalable response prediction for display advertising. ACM TIST, Vol. 5, 4 (2015), 61:1--61:34.

Digital Library

[4]

Miroslav Dud'ik, John Langford, and Lihong Li. 2011. Doubly robust policy evaluation and learning. In ICML .

[5]

Ali Mamdouh Elkahky, Yang Song, and Xiaodong He. 2015. A multi-view deep learning approach for cross domain user modeling in recommendation systems. In WWW .

[6]

Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In IJCAI .

[7]

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, and Joaquin Qui n onero Candela. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. In ADKDD .

[8]

James J. Heckman. 1979. Sample selection bias as a specification error. Econometrica, Vol. 47, 1 (1979), 153--161.

[9]

Daniel G. Horvitz and Donovan J. Thompson. 1952. A Generalization of Sampling Without Replacement From a Finite Universe. J. Amer. Statist. Assoc., Vol. 47 (1952), 663--685.

[10]

Cho-Jui Hsieh, Nagarajan Natarajan, and Inderjit Dhillon. 2015. PU Learning for Matrix Completion. In ICML .

[11]

Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In ICDM .

[12]

Machael Jahrer, Andreas Töscher, Jeong-Yoon Lee, Jingjing Deng, Hang Zhang, and Jacob Spoelstra. 2012. Ensemble of Collaborative Filtering and Feature Engineered Model for Click Through Rate Prediction. In KDD Cup 2012 Workshop .

[13]

Yuchin Juan, Damien Lefortier, and Olivier Chapelle. 2017. Field-aware factorization machines in a real-world online advertising system. In WWW .

[14]

Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware Factorization Machines for CTR Prediction. In RecSys .

[15]

Himabindu Lakkaraju, Jon Kleinberg, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2017. The selective labels problem: Evaluating algorithmic predictions in the presence of unobservables. In KDD .

[16]

Carolin Lawrence, Artem Sokolov, and Stefan Riezler. 2017. Counterfactual Learning from Bandit Feedback under Deterministic Logging: A Case Study in Statistical Machine Translation. In EMNLP .

[17]

Damien Lefortier, Adith Swaminathan, Xiaotao Gu, Thorsten Joachims, and Maarten de Rijke. 2016. Large-scale validation of counterfactual learning methods: A test-bed. In NIPS Workshop on Inference and Learning of Hypothetical and Counterfactual Interventions in Complex Systems .

[18]

Roderick J. A. Little and Donald B. Rubin. 2002. Statistical Analysis with Missing Data second ed.). Wiley-Interscience.

[19]

David C. Liu, Stephanie Rogers, Raymond Shiau, Dmitry Kislyuk, Kevin C. Ma, Zhigang Zhong, Jenny Liu, and Yushi Jing. 2017. Related Pins at Pinterest: The Evolution of a Real-World Recommender System. In WWW .

[20]

Benjamin M. Marlin and Richard S. Zemel. 2009. Collaborative Prediction and Ranking with Non-random Missing Data. In RecSys .

[21]

H. Brendan McMahan, Gary Holt, D. Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, and Jeremy Kubica. 2013. Ad Click Prediction: a View from the Trenches. In KDD .

[22]

Yusuke Narita, Shota Yasui, and Kohei Yata. 2019. Efficient Counterfactual Learning from Bandit Feedback. In AAAI .

[23]

Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data. ACM TOIS, Vol. 37 (2018), 5:1--5:35.

Digital Library

[24]

Steffen Rendle. 2010. Factorization machines. In ICDM .

[25]

Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: estimating the click-through rate for new ADs. In WWW .

[26]

Nir Rosenfeld, Yishay Mansour, and Elad Yom-Tov. 2017. Predicting counterfactuals from large historical data and small randomized trials. In WWW .

[27]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations As Treatments: Debiasing Learning and Evaluation. In ICML .

[28]

Adith Swaminathan and Thorsten Joachims. 2015. Batch learning from logged bandit feedback through counterfactual risk minimization. JMLR, Vol. 16 (2015), 1731--1755.

Digital Library

[29]

Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah Estrin. 2018. Unbiased offline recommender evaluation for missing-not-at-random implicit feedback. In RecSys .

[30]

Hsiang-Fu Yu, Mikhail Bilenko, and Chih-Jen Lin. 2017a. Selection of Negative Samples for One-class Matrix Factorization. In SDM .

[31]

Hsiang-Fu Yu, Hsin-Yuan Huang, Inderjit S. Dihillon, and Chih-Jen Lin. 2017b. A Unified Algorithm for One-class Structured Matrix Factorization with Side Information. In AAAI .

[32]

Bowen Yuan, Meng-Yuan Yang, Jui-Yang Hsia, Hong Zhu, Zhirong Liu, Zhenhua Dong, and Chih-Jen Lin. 2019. One-class Field-aware Factorization Machines for Recommender Systems with Implicit Feedbacks . Technical Report. National Taiwan University.

[33]

Weinan Zhang, Tianxiong Zhou, Jun Wang, and Jian Xu. 2016. Bid-aware gradient descent for unbiased learning with censored data in display advertising. In KDD .

[34]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD .

Cited By

Jian MBai YFu XGuo JShi GWu L(2024)Counterfactual Graph Convolutional Learning for Personalized RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365563215:4(1-20)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1145/3655632
Li WYao DGong CChu XJing QZhou XZhang YFan YBi J(2024)CausalTAD: Causal Implicit Generative Model for Debiased Online Trajectory Anomaly Detection2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00341(4477-4490)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00341
Zhang HWang SLi HZheng CChen XLiu LLuo SWu P(2024)Uncovering the Propensity Identification Problem in Debiased Recommendations2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00056(653-666)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00056
Show More Cited By

Index Terms

Improving Ad Click Prediction by Considering Non-displayed Events
1. Information systems
  1. Information systems applications
    1. Computational advertising

Recommendations

Unbiased Ad Click Prediction for Position-aware Advertising Systems
RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

Click-through rate (CTR) prediction is a core problem of building advertising systems. In many real-world applications, because an ad placed in various positions has different click probabilities, the position information should be considered in both ...
Improving click-through rate prediction accuracy in online advertising by transfer learning
WI '17: Proceedings of the International Conference on Web Intelligence

As the main revenue source of Internet companies, online advertising is always a significant topic, where click-through rate (CTR) prediction plays a central role. In online advertising systems, there are often many advertisement products. Due to the ...
Ad Click-Through Rate Prediction: A Survey
Database Systems for Advanced Applications. DASFAA 2021 International Workshops
Abstract
Ad click-through rate prediction (CTR), as an essential task of charging advertisers in the field of E-commerce, provides users with appropriate advertisements according to user interests to increase users’ click-through rate based on user clicks. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
863
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)10

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jian MBai YFu XGuo JShi GWu L(2024)Counterfactual Graph Convolutional Learning for Personalized RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365563215:4(1-20)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1145/3655632
Li WYao DGong CChu XJing QZhou XZhang YFan YBi J(2024)CausalTAD: Causal Implicit Generative Model for Debiased Online Trajectory Anomaly Detection2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00341(4477-4490)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00341
Zhang HWang SLi HZheng CChen XLiu LLuo SWu P(2024)Uncovering the Propensity Identification Problem in Debiased Recommendations2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00056(653-666)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00056
Chen XFeng SYang MZhao KXu RCui CChen M(2024)Modeling question difficulty for unbiased cognitive diagnosis: A causal perspectiveKnowledge-Based Systems10.1016/j.knosys.2024.111750294(111750)Online publication date: Jun-2024
https://doi.org/10.1016/j.knosys.2024.111750
Yang XLi XLiu ZWang YLu SLiu F(2024)Disentangled causal representation learning for debiasing recommendation with uniform dataApplied Intelligence10.1007/s10489-024-05497-954:8(6760-6775)Online publication date: 24-May-2024
https://doi.org/10.1007/s10489-024-05497-9
Li HXiao YZheng CWu PCui PKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Propensity mattersProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619239(20182-20194)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619239
Xu STan JHeinecke SLi VZhang Y(2023)Deconfounded Causal Collaborative FilteringACM Transactions on Recommender Systems10.1145/36060351:4(1-25)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1145/3606035
Wang CSun JDong ZZhu JLi ZLi RZhang R(2023)Data-free Knowledge Distillation for Reusing Recommendation ModelsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608789(386-395)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608789
Gao JHan SZhu HYang SJiang YXu JZheng BFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Rec4Ad: A Free Lunch to Mitigate Sample Selection Bias for Ads CTR Prediction in TaobaoProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615496(4574-4580)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615496
Zhao ZGao JZhang YHan SLou SSheng XWang ZZhu HJiang YXu JZheng BFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)COPR: Consistency-Oriented Pre-Ranking for Online AdvertisingProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615465(4974-4980)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615465
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents