research-article

What You Look Matters?: Offline Evaluation of Advertising Creatives for Cold-start Problem

Authors:

Weiying MaAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 2605 - 2613

https://doi.org/10.1145/3357384.3357813

Published: 03 November 2019 Publication History

Abstract

Modern online auction-based advertising systems combine item and user features to promote ad creatives with the most revenue.However, new ad creatives have to display for certain initial users before enough click statistics could collected and utilized in later ads ranking and bidding processes. This leads to a well-known challenging cold start problem.In this paper, we argue that the content of the creatives intrinsically determines their performance (e.g. ctr, cvr), and we add a pre-ranking stage based on the content. The stage prunes inferior creatives and thus makes online impressions more effective. Since the pre-ranking stage can be executed offline, we can use deep features and take their well generalization to navigate the cold start problem.Specifically, we propose Pre Evaluation Ad Creation Model (PEAC), a novel method to evaluate creatives even before they were shown in the online ads system. Our proposed PEAC only utilizes ads information such as verbal and visual content, but requires no user data as features. During the online A/B testing, PEAC shows significant improvement in revenue. The method has been implemented and deployed in the large scale online advertising system at ByteDance. Furthermore, we provide detailed analysis on what the model learns, which also gives suggestions for ad creative design.

References

[1]

Deepak Agarwal, Rahul Agrawal, Rajiv Khanna, and Nagaraj Kota. 2010. Estimating rates of rare events with multiple hierarchies through scalable log-linear models. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 213--222.

Digital Library

[2]

Javad Azimi, Ruofei Zhang, Yang Zhou, Vidhya Navalpakkam, Jianchang Mao, and Xiaoli Fern. 2012. The impact of visual appearance on user response in online display advertising. In Proceedings of the 21st International Conference on World Wide Web. ACM, 457--458.

Digital Library

[3]

Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2015. Learning to Rank using Gradient Descent. In ICML.

[4]

Deepayan Chakrabarti, Deepak Agarwal, and Vanja Josifovski. 2008. Contextual advertising by combining relevance with click feedback. In Proceedings of the 17th international conference on World Wide Web. ACM, 417--426.

Digital Library

[5]

Olivier Chapelle and Lihong Li. 2011. An Empirical Evaluation of Thompson Sampling. In NIPS.

[6]

Junxuan Chen, Baigui Sun, Hao Li, Hongtao Lu, and Xian-Sheng Hua. 2016. Deep ctr prediction in display advertising. In Proceedings of the 2016 ACM on Multimedia Conference. ACM, 811--820.

Digital Library

[7]

Haibin Cheng and Erick Cantú-Paz. 2010. Personalized click prediction in sponsored search. In Proceedings of the third ACM international conference on Web search and data mining. ACM, 351--360.

Digital Library

[8]

Haibin Cheng, Roelof van Zwol, Javad Azimi, Eren Manavoglu, Ruofei Zhang, Yang Zhou, and Vidhya Navalpakkam. 2012. Multimedia features for click prediction of new ads in display advertising. In KDD. ACM, 777--785.

[9]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et almbox. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 7--10.

Digital Library

[10]

Kushal S Dave and Vasudeva Varma. 2010. Learning the click-through rate for rare/new ads from similar ads. In SIGIR. ACM, 897--898.

[11]

Tiezheng Ge, Liqin Zhao, Guorui Zhou, Keyu Chen, Shuying Liu, Huiming Yi, Zelin Hu, Bochao Liu, Peng Sun, Haoyu Liu, et almbox. 2017a. Image Matters: Jointly Train Advertising CTR Model with Image Representation of Ad and User Behavior. (2017).

[12]

Tiezheng Ge, Liqin Zhao, Guorui Zhou, Keyu Chen, Shuying Liu, Huimin Yi, Zelin Hu, Bochao Liu, Peng Sun, Haoyu Liu, Pengtao Yi, Sui Huang, Zhiqiang Zhang, Xiaoqiang Zhu, Yu Zhang, and Kun Gai. 2017b. Image Matters: Visually modeling user behaviors using Advanced Model Server.

[13]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arXiv preprint arXiv:1512.03385 (2015).

[14]

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et almbox. 2014. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising. ACM, 1--9.

Digital Library

[15]

Deng J., Dong W., Socher R., Li L.-J., Li K., and Fei-Fei L. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.

[16]

Simonyan Karen and Zisserman Andrew. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition., Vol. abs/1409.1556 (2014).

[17]

H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et almbox. 2013. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1222--1230.

Digital Library

[18]

Kaixiang Mo, Bo Liu, Lei Xiao, Yong Li, and Jie Jiang. 2015. Image Feature Learning for Cold Start Problem in Display Advertising. In IJCAI.

[19]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]

Hochreiter Sepp and Schmidhuber Jürgen. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[21]

Hamed Valizadegan, Jin Rong, Ruofei Zhang, and Jianchang Mao. 2009. Learning to Rank by Optimizing NDCG Measure. In Advances in Neural Information Processing Systems 22, Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta (Eds.). Curran Associates, Inc., 1883--1891.

[22]

Liu Wei, Anguelov Dragomir, Erhan Dumitru, Szegedy Christian, Reed Scott, Fu Cheng-Yang, and Berg Alexander C. 2015. SSD: Single Shot MultiBox Detector.

[23]

Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level Convolutional Networks for Text Classification. In NIPS, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc., 649--657.

[24]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1059--1068.

Digital Library

Cited By

Yakovleva DTelnov SMakarov IFilchenkov A(2024)Bid Landscape Forecasting and Cold Start Problem With TransformersIEEE Access10.1109/ACCESS.2024.336049312(19117-19127)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3360493
Li GYang X(2024)Two-Stage Dynamic Creative Optimization Under Sparse Ambiguous Samples for e-Commerce AdvertisingSN Computer Science10.1007/s42979-024-03332-z5:8Online publication date: 26-Oct-2024
https://doi.org/10.1007/s42979-024-03332-z
Perdana ABharathi S(2023)Bytedance’s global rise: An exemplar of strategic adaptability and digital innovation in a global contextJournal of Information Technology Teaching Cases10.1177/20438869231211598Online publication date: 30-Oct-2023
https://doi.org/10.1177/20438869231211598
Show More Cited By

Index Terms

What You Look Matters?: Offline Evaluation of Advertising Creatives for Cold-start Problem
1. Information systems
  1. Information systems applications
    1. Computational advertising
  2. World Wide Web
    1. Online advertising
      1. Display advertising
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Algorithmic game theory and mechanism design
      1. Computational advertising theory

Recommendations

A framework for intermediated online targeted advertising with banner ranking mechanism

Reinforced by the fast growth of electronic commerce, even during the current global economic downturn, intermediated online targeted advertising (IOTA) has emerged as a promising electronic business model empowered by the Web 2.0 principle. IOTA ...
Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Click-through rate (CTR) prediction is one of the most central tasks in online advertising systems. Recent deep learning-based models that exploit feature embedding and high-order data nonlinearity have shown dramatic successes in CTR prediction. However,...
What matters to users?: factors that affect users' willingness to share information with online advertisers
SOUPS '13: Proceedings of the Ninth Symposium on Usable Privacy and Security

Much of the debate surrounding online behavioral advertising (OBA) has centered on how to provide users with notice and choice. An important element left unexplored is how advertising companies' privacy practices affect users' attitudes toward data ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
486
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yakovleva DTelnov SMakarov IFilchenkov A(2024)Bid Landscape Forecasting and Cold Start Problem With TransformersIEEE Access10.1109/ACCESS.2024.336049312(19117-19127)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3360493
Li GYang X(2024)Two-Stage Dynamic Creative Optimization Under Sparse Ambiguous Samples for e-Commerce AdvertisingSN Computer Science10.1007/s42979-024-03332-z5:8Online publication date: 26-Oct-2024
https://doi.org/10.1007/s42979-024-03332-z
Perdana ABharathi S(2023)Bytedance’s global rise: An exemplar of strategic adaptability and digital innovation in a global contextJournal of Information Technology Teaching Cases10.1177/20438869231211598Online publication date: 30-Oct-2023
https://doi.org/10.1177/20438869231211598
You SWang CWu BLiu JLu QHan GXiao Y(2023)What Image do You Need? A Two-stage Framework for Image Selection in E-commerceCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3584646(452-456)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543873.3584646
Mu CTang XLuo JLiu Y(2023)GMiRec: A Multi-image Visual Recommendation Model Based on a Gated Neural NetworkKnowledge Science, Engineering and Management10.1007/978-3-031-40286-9_27(331-342)Online publication date: 9-Aug-2023
https://doi.org/10.1007/978-3-031-40286-9_27
Ko JJeong JKim K(2023)Contrastive Learning for Topic-Dependent Image RankingRecommender Systems in Fashion and Retail10.1007/978-3-031-22192-7_5(79-98)Online publication date: 2-Mar-2023
https://doi.org/10.1007/978-3-031-22192-7_5
Yu TJin ZLiu JYang YFei HLi P(2022)Boost CTR Prediction for New Advertisements via Modeling Visual Content2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020786(2140-2149)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020786
Xiong WXiong ZTian T(2022)Who to show the ad to? Behavioral targeting in Internet advertisingJournal of Internet and Digital Economics10.1108/JIDE-12-2021-00232:1(15-26)Online publication date: 21-Mar-2022
https://doi.org/10.1108/JIDE-12-2021-0023
Wang SLiu QGe TLian DZhang Z(2021)A Hybrid Bandit Model with Visual Priors for Creative Ranking in Display AdvertisingProceedings of the Web Conference 202110.1145/3442381.3449910(2324-2334)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449910

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten