research-article

Uncovering Bias in Ad Feedback Data Analyses & Applications✱

Authors:

Mounia LalmasAuthors Info & Claims

WWW '19: Companion Proceedings of The 2019 World Wide Web Conference

Pages 614 - 623

https://doi.org/10.1145/3308560.3317304

Published: 13 May 2019 Publication History

Abstract

Electronic publishers and other web-companies are starting to collect user feedback on ads with the aim of using this signal to maintain the quality of ads shown on their sites. However, users are not randomly sampled to provide feedback on ads, but targeted. Furthermore some users who provide feedback may be prone to dislike ads more than the general user. This raises questions about the reliability of ad feedback as a signal for measuring ad quality and whether it can be used in ad ranking. In this paper we start by gaining insights to such signals by analyzing the feedback event logs attributed to users of a popular mobile news app. We then propose a model to reduce potential biases in ad feedback data. Finally, we conclude by comparing the effectiveness of reducing the bias in ad feedback data using existing ad ranking methods along with a new and novel approach we propose that takes revenue considerations into account.

References

[1]

M. Aharon, N. Aizenberg, E. Bortnikov, R. Lempel, R. Adadi, T. Benyamini, L. Levin, R. Roth, and O. Serfaty. Off-set: one-pass factorization of feature sets for online recommendation in persistent cold start settings. In RECSYS’13, pages 375–378. ACM, 2013.

Digital Library

[2]

A. Ashkan and C. L. A. Clarke. Modeling browsing behavior for click analysis in sponsored search. In CIKM, 2012.

Digital Library

[3]

S. Athey and D. Nekipelov. A structural model of sponsored search advertising auctions. In Sixth ad auctions workshop, volume 15, 2010.

[4]

J. Azimi, R. Zhang, Y. Zhou, V. Navalpakkam, J. Mao, and X. Fern. Visual appearance of display ads and its effect on click through rate. In CIKM, 2012.

Digital Library

[5]

N. Barbieri, F. Silvestri, and M. Lalmas. Improving post-click user engagement on native ads via survival analysis. In WWW’16, pages 761–770, 2016.

Digital Library

[6]

H. Becker, A. Broder, E. Gabrilovich, V. Josifovski, and B. Pang. What happens after an ad click?: Quantifying the impact of landing pages in web advertising. In CIKM’09, 2009.

Digital Library

[7]

G. Brajnik and S. Gabrielli. A review of online advertising effects on the user experience. IJHCI, 26(10):971–997, 2010.

[8]

A. Z. Broder, M. Ciaramita, M. Fontoura, E. Gabrilovich, V. Josifovski, D. Metzler, V. Murdock, and V. Plachouras. To swing or not to swing: learning when (not) to advertise. In CIKM’08, 2008.

Digital Library

[9]

J. M. Chambers and T. J. Hastie. Statistical models in S. CRC Press, Inc., 1991.

Digital Library

[10]

H. Cheng, R. v. Zwol, J. Azimi, E. Manavoglu, R. Zhang, Y. Zhou, and V. Navalpakkam. Multimedia features for click prediction of new ads in display advertising. In SIGKDD’12, pages 777–785. ACM, 2012.

Digital Library

[11]

Y. Choi, M. Fontoura, E. Gabrilovich, V. Josifovski, M. Mediano, and B. Pang. Using landing pages for sponsored search ad selection. In WWW’10, 2010.

Digital Library

[12]

H. Cramer. Effects of ad quality & content-relevance on perceived content quality. In CHI ’15, pages 2231–2234, New York, NY, USA, 2015. ACM.

Digital Library

[13]

D. G. Goldstein, R. P. McAfee, and S. Suri. The cost of annoying ads. In WWW’13, 2013.

Digital Library

[14]

M. Grbovic, N. Djuric, V. Radosavljevic, F. Silvestri, R. Baeza-Yates, A. Feng, E. Ordentlich, L. Yang, and G. Owens. Scalable semantic matching of queries to ads in sponsored search advertising. In SIGIR’16, pages 375–384, 2016.

Digital Library

[15]

H. Hohnhold, D. O’Brien, and D. Tang. Focusing on the long-term: It’s good for users and business. In KDD’15, pages 1849–1858, 2015.

Digital Library

[16]

D. W. Hosmer Jr and S. Lemeshow. Applied logistic regression. John Wiley & Sons, 2004.

[17]

S. Ieong, M. Mahdian, and S. Vassilvitskii. Advertising in a stream. In WWW’14, 2014.

Digital Library

[18]

A. Kae, K. Kan, V. K. Narayanan, and D. Yankov. Categorization of display ads using image and landing page features. In LDMTA, 2011.

Digital Library

[19]

D. Kempe and B. Lucier. User satisfaction in competitive sponsored search. In WWW’14, 2014.

Digital Library

[20]

M. Lalmas, J. Lehmann, G. Shaked, F. Silvestri, and G. Tolomei. Promoting positive post-click experience for in-stream yahoo gemini users.

Digital Library

[21]

M. Lalmas, J. Lehmann, G. Shaked, F. Silvestri, and G. Tolomei. Promoting positive post-click experience for in-stream yahoo gemini users. In SIGKDD’15, pages 1929–1938, 2015.

Digital Library

[22]

R. J. Oentaryo, E.-P. Lim, J.-W. Low, D. Lo, and M. Finegold. Predicting response in mobile advertising with hierarchical importance-aware factorization machine. In WSDM’14, 2014.

Digital Library

[23]

R. Rosales, H. Cheng, and E. Manavoglu. Post-click conversion modeling and analysis for non-guaranteed delivery display advertising. In WSDM’12, 2012.

Digital Library

[24]

D. Sculley, R. G. Malkin, S. Basu, and R. J. Bayardo. Predicting bounce rates in sponsored search advertisements. In KDD’09, 2009.

Digital Library

[25]

G. Shmueli. To explain or to predict? Statistical science, pages 289–310, 2010.

[26]

E. Sodomka, S. Lahaie, and D. Hillard. A predictive model for advertiser value-per-click in sponsored search. In WWW’13, 2013.

Digital Library

[27]

F. Vella. Estimating models with sample selection bias: a survey. Journal of Human Resources, pages 127–169, 1998.

[28]

T. Wang, J. Bian, S. Liu, Y. Zhang, and T.-Y. Liu. Psychological advertising: exploring user psychology for click prediction in sponsored search. In SIGKDD’13, 2013.

Digital Library

[29]

D. Yin, B. Cao, J.-T. Sun, and B. D. D. 0001. Estimating ad group performance in sponsored search. In WSDM’14, 2014.

Digital Library

[30]

K. Zhou, M. Redi, A. Haines, and M. Lalmas. Predicting pre-click quality for native advertisements. In WWW’16, pages 299–310, 2016.

Digital Library

Cited By

Mena-Maldonado ECañamares RCastells PRen YSanderson M(2021)Popularity Bias in False-positive Metrics for Recommender Systems EvaluationACM Transactions on Information Systems10.1145/345274039:3(1-43)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3452740
Zendel OFyodorov YRaiber FSilberstein NSomekh OTabaja Ad'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)Leveraging User Email Actions to Improve Ad-Close PredictionProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3412093(2293-2296)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3412093
Silberstein NSomekh OKoren YAharon MPorat DShahar AWu TCaverlee JHu XLalmas MWang W(2020)Ad Close Mitigation for Improved User Experience in Native AdvertisementsProceedings of the 13th International Conference on Web Search and Data Mining10.1145/3336191.3371798(546-554)Online publication date: 20-Jan-2020
https://dl.acm.org/doi/10.1145/3336191.3371798

Index Terms

Uncovering Bias in Ad Feedback Data Analyses & Applications✱
1. Applied computing
  1. Law, social and behavioral sciences
2. Information systems
  1. Information systems applications
  2. World Wide Web

Index terms have been assigned to the content through auto-classification.

Recommendations

OLSRv2 implementation and performance evaluation with link layer feedback
IWCMC '07: Proceedings of the 2007 international conference on Wireless communications and mobile computing

OLSRv2 is being standardized in the IETF MANET (Mobile Ad hoc Network) working group. OLSRv2 uses the common packet format, that is also being standardized in the MANET WG. It is necessary to validate the OLSRv2 protocol specification by implementing ...
On bias problem in relevance feedback
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

Relevance feedback is an effective approach to improve retrieval quality over the initial query. Typical relevance feedback methods usually select top-ranked documents for relevance judgments, then query expansion or model updating are carried out based ...
Mobile data collector strategy for delay-sensitive applications over wireless sensor networks

Applications that require fast response time such as emergency preparedness and hostile environment surveillance pose challenging obstacles to wireless sensor network (WSN) protocols. A routing protocol must provide fast and reliable techniques for data ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '19: Companion Proceedings of The 2019 World Wide Web Conference

May 2019

1331 pages

ISBN:9781450366755

DOI:10.1145/3308560

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
183
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mena-Maldonado ECañamares RCastells PRen YSanderson M(2021)Popularity Bias in False-positive Metrics for Recommender Systems EvaluationACM Transactions on Information Systems10.1145/345274039:3(1-43)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3452740
Zendel OFyodorov YRaiber FSilberstein NSomekh OTabaja Ad'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)Leveraging User Email Actions to Improve Ad-Close PredictionProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3412093(2293-2296)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3412093
Silberstein NSomekh OKoren YAharon MPorat DShahar AWu TCaverlee JHu XLalmas MWang W(2020)Ad Close Mitigation for Improved User Experience in Native AdvertisementsProceedings of the 13th International Conference on Web Search and Data Mining10.1145/3336191.3371798(546-554)Online publication date: 20-Jan-2020
https://dl.acm.org/doi/10.1145/3336191.3371798

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents