research-article

Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction

Authors:

Bo ZhengAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 369 - 379

https://doi.org/10.1145/3485447.3511965

Published: 25 April 2022 Publication History

Abstract

Alleviating the delayed feedback problem is of crucial importance for the conversion rate(CVR) prediction in online advertising. Previous delayed feedback modeling methods using an observation window to balance the trade-off between waiting for accurate labels and consuming fresh feedback. Moreover, to estimate CVR upon the freshly observed but biased distribution with fake negatives, the importance sampling is widely used to reduce the distribution bias. While effective, we argue that previous approaches falsely treat fake negative samples as real negative during the importance weighting and have not fully utilized the observed positive samples, leading to suboptimal performance.

In this work, we propose a new method, DElayed Feedback modeling with UnbiaSed Estimation, (DEFUSE), which aim to respectively correct the importance weights of the immediate positive, the fake negative, the real negative, and the delay positive samples at finer granularity. Specifically, we propose a two-step optimization approach that first infers the probability of fake negatives among observed negatives before applying importance sampling. To fully exploit the ground-truth immediate positives from the observed distribution, we further develop a bi-distribution modeling framework to jointly model the unbiased immediate positives and the biased delay conversions. Experimental results on both public and our industrial datasets validate the superiority of DEFUSE. Codes are available at https://github.com/ychen216/DEFUSE.git.

References

[1]

Christopher M Bishop. 2006. Pattern recognition. Machine learning 128, 9 (2006).

[2]

Olivier Chapelle. 2014. Modeling delayed feedback in display advertising. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA - August 24 - 27, 2014, Sofus A. Macskassy, Claudia Perlich, Jure Leskovec, Wei Wang, and Rayid Ghani (Eds.). ACM, 1097–1105. https://doi.org/10.1145/2623330.2623634

Digital Library

[3]

David S Evans. 2009. The online advertising industry: Economics, evolution, and privacy. Journal of economic perspectives 23, 3 (2009), 37–60.

[4]

Avi Goldfarb and Catherine Tucker. 2011. Online display advertising: Targeting and obtrusiveness. Marketing Science 30, 3 (2011), 389–404.

Digital Library

[5]

Siyu Gu, Xiang-Rong Sheng, Ying Fan, Guorui Zhou, and Xiaoqiang Zhu. 2021. Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling. In KDD ’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, Feida Zhu, Beng Chin Ooi, and Chunyan Miao (Eds.). ACM, 2890–2898. https://doi.org/10.1145/3447548.3467086

Digital Library

[6]

Amélie Héliou, Panayotis Mertikopoulos, and Zhengyuan Zhou. 2020. Gradient-free Online Learning in Games with Delayed Rewards. In ICML’20: The 37th International Conference on Machine Learning.

[7]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015(JMLR Workshop and Conference Proceedings, Vol. 37), Francis R. Bach and David M. Blei (Eds.). JMLR.org, 448–456. http://proceedings.mlr.press/v37/ioffe15.html

[8]

Pooria Joulani, Andras Gyorgy, and Csaba Szepesvári. 2013. Online learning under delayed feedback. In International Conference on Machine Learning. PMLR, 1453–1461.

[9]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6980

[10]

Sofia Ira Ktena, Alykhan Tejani, Lucas Theis, Pranay Kumar Myana, Deepak Dilipkumar, Ferenc Huszár, Steven Yoo, and Wenzhe Shi. 2019. Addressing delayed feedback for continuous training with neural networks in CTR prediction. In Proceedings of the 13th ACM Conference on Recommender Systems, RecSys 2019, Copenhagen, Denmark, September 16-20, 2019, Toine Bogers, Alan Said, Peter Brusilovsky, and Domonkos Tikk (Eds.). ACM, 187–195. https://doi.org/10.1145/3298689.3347002

Digital Library

[11]

Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. In The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’12, Beijing, China, August 12-16, 2012, Qiang Yang, Deepak Agarwal, and Jian Pei (Eds.). ACM, 768–776. https://doi.org/10.1145/2339530.2339651

Digital Library

[12]

Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 768–776.

[13]

Xun Liu, Wei Xue, Lei Xiao, and Bo Zhang. 2017. PBODL : Parallel Bayesian Online Deep Learning for Click-Through Rate Prediction in Tencent Advertising System. CoRR abs/1707.00802(2017). arXiv:1707.00802http://arxiv.org/abs/1707.00802

[14]

Quan Lu, Shengjun Pan, Liang Wang, Junwei Pan, Fengdan Wan, and Hongxia Yang. 2017. A practical framework of conversion rate prediction for online display advertising. In Proceedings of the ADKDD’17. 1–9.

Digital Library

[15]

Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. 2018. Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, Yike Guo and Faisal Farooq (Eds.). ACM, 1930–1939. https://doi.org/10.1145/3219819.3220007

Digital Library

[16]

Andrew L Maas, Awni Y Hannun, Andrew Y Ng, 2013. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, Vol. 30. Citeseer, 3.

[17]

Travis Mandel, Yun-En Liu, Emma Brunskill, and Zoran Popović. 2015. The queue method: Handling delay, heuristics, prior data, and evaluation in bandits. In Twenty-Ninth AAAI Conference on Artificial Intelligence.

Digital Library

[18]

Taesup Moon, Lihong Li, Wei Chu, Ciya Liao, Zhaohui Zheng, and Yi Chang. 2010. Online learning for recency search ranking using real-time user feedback. In Proceedings of the 19th ACM Conference on Information and Knowledge Management, CIKM 2010, Toronto, Ontario, Canada, October 26-30, 2010, Jimmy Huang, Nick Koudas, Gareth J. F. Jones, Xindong Wu, Kevyn Collins-Thompson, and Aijun An (Eds.). ACM, 1501–1504. https://doi.org/10.1145/1871437.1871657

Digital Library

[19]

Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, and Aaron Flores. 2019. Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis (Eds.). ACM, 2689–2697. https://doi.org/10.1145/3292500.3330783

Digital Library

[20]

Ciara Pike-Burke, Shipra Agrawal, Csaba Szepesvari, and Steffen Grunewalder. 2018. Bandits with delayed, aggregated anonymous feedback. In International Conference on Machine Learning. PMLR, 4105–4113.

[21]

Doyen Sahoo, Quang Pham, Jing Lu, and Steven C. H. Hoi. 2018. Online Deep Learning: Learning Deep Neural Networks on the Fly. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, Jérôme Lang (Ed.). ijcai.org, 2660–2666. https://doi.org/10.24963/ijcai.2018/369

[22]

Linqi Song, Cem Tekin, and Mihaela van der Schaar. 2016. Online Learning in Large-Scale Contextual Recommender Systems. IEEE Trans. Serv. Comput. 9, 3 (2016), 433–445. https://doi.org/10.1109/TSC.2014.2365795

[23]

Yumin Su, Liang Zhang, Quanyu Dai, Bo Zhang, Jinyao Yan, Dan Wang, Yongjun Bao, Sulong Xu, Yang He, and Weipeng Yan. 2020. An Attention-based Model for Conversion Rate Prediction with Delayed Feedback via Post-click Calibration. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, Christian Bessiere (Ed.). ijcai.org, 3522–3528. https://doi.org/10.24963/ijcai.2020/487

[24]

Claire Vernade, Olivier Cappé, and Vianney Perchet. 2017. Stochastic bandit models for delayed conversions. arXiv preprint arXiv:1706.09186(2017).

[25]

Yanshi Wang, Jie Zhang, Qing Da, and Anxiang Zeng. 2020. Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction. CoRR abs/2011.11826(2020). arXiv:2011.11826https://arxiv.org/abs/2011.11826

[26]

Zhuojian Xiao, Yunjiang Jiang, Guoyu Tang, Lin Liu, Sulong Xu, Yun Xiao, and Weipeng Yan. 2021. Adversarial Mixture Of Experts with Category Hierarchy Soft Constraint. In 37th IEEE International Conference on Data Engineering, ICDE 2021, Chania, Greece, April 19-22, 2021. IEEE, 2453–2463. https://doi.org/10.1109/ICDE51399.2021.00278

[27]

Jia-Qi Yang, Xiang Li, Shuguang Han, Tao Zhuang, De-Chuan Zhan, Xiaoyi Zeng, and Bin Tong. 2021. Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021. AAAI Press, 4582–4589. https://ojs.aaai.org/index.php/AAAI/article/view/16587

[28]

Shota Yasui, Gota Morishita, Komei Fujita, and Masashi Shibata. 2020. A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback. In WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen (Eds.). ACM / IW3C2, 2740–2746. https://doi.org/10.1145/3366423.3380032

Digital Library

[29]

Yuya Yoshikawa and Yusaku Imai. 2018. A Nonparametric Delayed Feedback Model for Conversion Rate Prediction. CoRR abs/1802.00255(2018). arXiv:1802.00255http://arxiv.org/abs/1802.00255

[30]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059–1068.

Digital Library

Cited By

Dai SShao NZhu JZhang XDong ZXu JDai QWen J(2024)Modeling User Attention in Music Recommendation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00064(761-774)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00064
Zhao YYan XGui XHan SSheng XYu GChen JXu ZZheng BFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Entire Space Cascade Delayed Feedback Modeling for Effective Conversion Rate PredictionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615475(4981-4987)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615475
Dai SZhou YXu JWen JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Dually Enhanced Delayed Feedback Modeling for Streaming Conversion Rate PredictionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614856(390-399)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614856
Show More Cited By

Index Terms

Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
2. Information systems
  1. Information systems applications
    1. Computational advertising

Index terms have been assigned to the content through auto-classification.

Recommendations

A Multi-Task Learning Approach for Delayed Feedback Modeling
WWW '22: Companion Proceedings of the Web Conference 2022

Conversion rate (CVR) prediction is one of the most essential tasks for digital display advertising. In industrial recommender systems, online learning is particularly favored for its capability to capture the dynamic change of data distribution, which ...
Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

One of the difficulties of conversion rate (CVR) prediction is that the conversions can delay and take place long after the clicks. The delayed feedback poses a challenge: fresh data are beneficial to continuous training but may not have complete label ...
Unbiased Delayed Feedback Label Correction for Conversion Rate Prediction
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Conversion rate prediction is critical to many online applications such as digital display advertising. To capture dynamic data distribution, industrial systems often require retraining models on recent data daily or weekly. However, the delay of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
289
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dai SShao NZhu JZhang XDong ZXu JDai QWen J(2024)Modeling User Attention in Music Recommendation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00064(761-774)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00064
Zhao YYan XGui XHan SSheng XYu GChen JXu ZZheng BFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Entire Space Cascade Delayed Feedback Modeling for Effective Conversion Rate PredictionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615475(4981-4987)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615475
Dai SZhou YXu JWen JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Dually Enhanced Delayed Feedback Modeling for Streaming Conversion Rate PredictionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614856(390-399)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614856
Zhang YHua HGuo HWang SZhong CZhang SFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)3MN: Three Meta Networks for Multi-Scenario and Multi-Task Learning in Online Advertising Recommender SystemsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614651(4945-4951)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614651
Chan ZZhang YHan SBai YSheng XLou SHu JLiu BJiang YXu JZheng BSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Capturing Conversion Rate Fluctuation during Sales Promotions: A Novel Historical Data Reuse ApproachProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599788(3774-3784)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599788
Wang YSun PZhang MJia QLi JMa SSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Unbiased Delayed Feedback Label Correction for Conversion Rate PredictionProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599536(2456-2466)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599536
Liu QLi HAo XGuo YDong ZZhang RChen QTong JHe QChen HDuh WHuang HKato MMothe JPoblete B(2023)Online Conversion Rate Prediction via Neural Satellite Networks in Delayed Feedback AdvertisingProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591747(1406-1415)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591747
Zheng XWang ZChen CZhu FQian J(2023)Freshness or Accuracy, Why Not Both? Addressing Delayed Feedback via Dynamic Graph Neural Networks2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00059(405-414)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00059
Guo YLi HAo XLu MLiu DXiao LJiang JHe QAl Hasan MXiong L(2022)Calibrated Conversion Rate Prediction via Knowledge Distillation under Delayed Feedback in Online AdvertisingProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557557(3983-3987)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557557

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents