Abstract
Digital algorithms try to display content that engages consumers. To do this, algorithms need to overcome a ‘cold-start problem’ by swiftly learning whether content engages users. This requires feedback from users. The algorithm targets segments of users. However, if there are fewer individuals in a targeted segment of users, simply because this group is rarer in the population, this could lead to uneven outcomes for minority relative to majority groups. This is because individuals in a minority segment are proportionately more likely to be test subjects for experimental content that may ultimately be rejected by the platform. We explore in the context of ads that are displayed following searches on Google whether this is indeed the case. Previous research has documented that searches for names associated in a US context with Black people on search engines were more likely to return ads that highlighted the need for a criminal background check than was the case for searches for white people. We implement search advertising campaigns that target ads to searches for Black and white names. Our ads are indeed more likely to be displayed following a search for a Black name, even though the likelihood of clicking was similar. Since Black names are less common, the algorithm learns about the quality of the underlying ad more slowly. As a result, an ad is more likely to persist for searches next to Black names than next to white names. Proportionally more Black name searches are likely to have a low-quality ad shown next to them, even though eventually the ad will be rejected. A second study where ads are placed following searches for terms related to religious discrimination confirms this empirical pattern. Our results suggest that as a practical matter, real-time algorithmic learning can lead minority segments to be more likely to see content that will ultimately be rejected by the algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Statements and Declarations
This research was not funded by any companies or any external grant. The authors have no competing interests to declare that are relevant to the content of this article. The authors have no financial or proprietary interests in any material discussed in this article. However, both authors have consulted widely outside of this research. Catherine Tucker’s conflict of interest statement may be found at https://mitmgmtfaculty.mit.edu/cetucker/disclosure/. Anja Lambrecht’s disclosure statement may be found at https://www.london.edu/faculty-and-research/faculty-profiles/l/lambrecht-a.
Notes
Throughout, the discussion in Hellman (2020) emphasizes that when considering algorithmic fairness, one should worry about the proportion – not the absolute number – of individuals that may be disadvantaged in a majority or minority group.
Sweeney (2013) also added Latanya and Latisha to the list based on observational data.
We started with 20 names and dropped any last names which were over 90% Hispanic in origin to avoid drawing in names most characteristic of another minority group. Table 8 in the appendix documents this. This procedure left us with 14 individual last names.
Anne Moore has over 1 million instagram followers https://www.instagram.com/itsannemoore/?hl=en. Tyrone Davis was an American blues singer https://en.wikipedia.org/wiki/Tyrone_Davis and Allison Williams is an actress https://en.wikipedia.org/wiki/Allison_Williams_(actress)
1990 was the last year we could find this data for. There were 7 first names where there was no frequency data, of which 6 were Black names. This suggests that these names were unusual or novel enough to have not been counted in the frequency tabulations of the 1990 census exercise.
For our discussion of average click-through rates, we always compute campaign-level click-through rates for campaigns that have more than zero impressions and then average those campaign-level click-through rates to obtain an average across campaigns. This approach is appropriate since the algorithm optimizes each campaign separately. If we compute the ratio of total number of impressions/total number of clicks by whether a search was for a white name or a Black name, this gives us an overall CTR of 0.007 for white-name searches and 0.011 for Black-name searches.
Such data on estimated first page bids has previously been used to understand price patterns in online search (Goldfarb & Tucker, 2011).
Google did not provide an estimate for the search term Hakim Miller.
We focus on records where there was data on at least one competitor available of the type that the respective test analyzes.
References
Abu-Elyounes, D. (2020). Contextual fairness: A legal and policy analysis of algorithmic fairness. University of Illinois Journal of Law, Technology, & Policy, 1.
Acquisti, A., & Fong, C. (2020). An experiment in hiring discrimination via online social networks. Management Science, 66(3), 1005–1024.
Ali, M., Sapiezynski, P., Bogen, M., Korolova, A., Mislove, A., & Rieke, A. (2019, nov). Discrimination through optimization: How facebook’s ad delivery can lead to biased outcomes. Proc ACM Hum-Comput Interact, 3(CSCW).
Ali, M., Sapiezynski, P., Korolova, A., Mislove, A., & Rieke, A. (2021). Ad delivery algorithms: The hidden arbiters of political messaging. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM ’21, New York, NY, USA (pp. 13–21). Association for Computing Machinery.
Barocas, S., Hardt, M., Narayanan, A. (2017). Fairness in machine learning. NIPS Tutorial, 1.
Barocas, S., & Selbst, A. D. (2016). Big data’s disparate impact. Californian Law Review, 104, 671.
Bertrand, M., & Mullainathan, S. (2004). Are emily and greg more employable than lakisha and jamal? a field experiment on labor market discrimination. American Economic Review, 94(4), 991–1013.
Burroughs, V. J., Maxey, R. W., & Levy, R. A. (2002). Racial and ethnic differences in response to medicines: towards individualized pharmaceutical treatment. Journal of the National Medical Association, 94(10 Suppl), 1.
Claussen, J., Peukert, C., & Sen, A. (2024). The editor and the algorithm: Returns to data and externalities in online news. Management Science, 0(0), 0.
Cowgill, B. (2017). Automating judgement and decision-making: Theory and evidence from résumé screening. In Columbia University, 2015 Empirical Management Conference.
Cowgill, B. (2018). The impact of algorithms on judicial discretion: Evidence from regression discontinuities. Technical report, Working paper.
Dastin, J. (2018). Amazon scraps secret ai recruiting tool that showed bias against women. San Fransico, CA: Reuters. Retrieved on October, 9, 2018.
Datta, A., Tschantz, M. C., & Datta, A. (2015). Automated experiments on ad privacy settings. Proceedings on Privacy Enhancing Technologies, 2015(1), 92–112.
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1).
Edelman, B., Ostrovsky, M., & Schwarz, M. (2007). Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. American Economic Review, 97(1), 242–259.
Fryer, R. G., Jr., & Levitt, S. D. (2004). The causes and consequences of distinctively black names. The Quarterly Journal of Economics, 119(3), 767–805.
Ghose, A., & Yang, S. (2009). An empirical analysis of search engine advertising: Sponsored search in electronic markets. Management Science, 55(10), 1605–1622.
Goldfarb, A., & Tucker, C. (2011). Search engine advertising: Channel substitution when pricing ads to context. Management Science, 57(3), 458–470.
Hellman, D. (2020). Measuring algorithmic fairness. Virginia Law Review, 106, 811.
Kleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015). Prediction policy problems. American Economic Review, 105(5), 491–95.
Lambrecht, A., & Tucker, C. (2019). Algorithmic bias? an empirical study of apparent gender-based discrimination in the display of stem career ads. Management Science, 65(7), 2966–2981.
Mitchell, S., Potash, E., Barocas, S., D’Amour, A., & Lum, K. (2021). Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8.
Nachbar, T. B. (2020). Algorithmic fairness, algorithmic discrimination. Florida State University Law Review, 48, 509.
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453.
Rutz, O. J., & Bucklin, R. E. (2011). From generic to branded: A model of spillover dynamics in paid search advertising. Journal of Marketing Research, 48(1), 87–102.
Srinivasan, R., & Sarial-Abi, G. (2021). When algorithms fail: Consumers’ responses to brand harm crises caused by algorithm errors. Journal of Marketing, 85(5).
Sweeney, L. (2013). Discrimination in online ad delivery. ACMQueue, 11(3), 10.
Tunuguntla, S., & Hoban, P. R. (2021). A near-optimal bidding strategy for real-time display advertising auctions. Journal of Marketing Research, 58(1), 1–21.
Ukanwa, K., & Rust, R. T. (2021). Algorithmic discrimination in service.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Thank you to NSF CAREER Award 6923256 and the Research and Materials Development Fund at London Business School for financial support. All errors are our own. We thank for their helpful comments: Garrett Johnson, Thomas Otter, Caroline Wiertz, Hema Yoganarasimhan and Xu Zhang; seminar participants at the eQSM virtual seminar, National University Singapore, IDC Herzliya, Rotterdam School of Management, the University of Chicago and the VIDE (Virtual Digital Economics Seminar); and participants at the 2020 Marketing Science conference. We are grateful to Chaoran Liu for excellent research support.
Appendix
Appendix
1.1 A.1 Recap of results by Sweeney (2013)
Percent of public record ads displayed in response to Black-name and white-name searches
Figure 9 shows the percentage of Black- and white-name searches in response to which public record ads were displayed in Sweeney (2013)’s original research (based on Figure 16 in the paper). Though Sweeney (2013) also discusses the distribution of ads on Reuters, we focus in this research on Google search ads so this figure reports the results for Google only. It is clear that the probability of a public record ad being displayed was higher for Black-name searches than for white-name searches.
1.2 A.2 Additional tables
Here, we report additional Appendix Tables that the main paper refers to, including tables on the names used and robustness checks of the empirical results when excluding observations with zero impressions.
1.3 A.3 Insights from competitive intelligence
One motivation of our study was Sweeney (2013) who had demonstrated that ads for background checking services were more likely to be shown following searches for Black than for white names. In our study, we purposely did not show an ad for background checking services, but uncovered similar patterns for a different type of ad. However, we can use the data that Google reports on competitive bidders to shed light on the extent to which background checking services bid for ads towards Black or white names when we advertised.
Google reports to advertisers how often specific competitors’ ads were shown alongside their ad. Figure 10 shows a screenshot as an example. We collected such information on other advertisers who were bidding on that keyword for each of our campaigns. This set of analyses focuses on campaigns where the number of impressions was large enough for Google to report what they refer to as an ‘auction insight report.’ As a result, 113 campaigns targeting Black-name searches and 27 campaigns targeting white-name searches that had low search activity during the campaigns are excluded from our analysis.
First, we study the extent to which public record companies compete with our campaign.Footnote 14 We find on average across campaigns, 2.5 such competitors for Black-name searches and 2.0 for white-name searches (N \(=\) 726, t \(=\) 3.52, P < 0.001). This difference in the number of competing public record companies is reflected in the overall number of competitors recorded for a name. White-name searches have on average 3.3 and Black-name searches 3.8 competitors (N \(=\) 546, t \(=\) 3.14, P < 0.002). The number of competing advertisers that are not public record companies is not significantly different (0.610 for white-name and 0.561 for Black-name searches, N \(=\) 546, t \(=\) 0.72, P \(=\) 0.473).
Second, we study the share of impressions that across campaigns goes to each of the public record companies that advertise. Again, we find that for Black-name searches, any of the public record companies that advertised had, on average, 18.3% of impressions, while for white-name searches these were 10.3% (N \(=\) 1630, t \(=\) 12.22, P < 0.001). Google does not provide precise information on impression shares less than 10%. Hence, we set the value for impression shares between 0 and 10% to 0.05. When alternatively using values of 0.01, of 0.09, or excluding those observations from the analysis, we similarly obtain that the share of impressions a public record company has when targeting Black-name searches is significantly higher than for white-name searches.
Information on competitive bidders reported by Google AdWords
Third, Google reports how much our campaigns overlapped with ads by competitors. We find that the average overlap rate with public record companies for Black-name searches was 27.2% and for white-name searches was 21.0% (N \(=\) 1630, t \(=\) 4.96, P < 0.001).
These results suggest that we observe a similar pattern of focus by background record companies, in that their ads are more likely to appear next to Black names, as documented by Sweeney (2013). We also checked the robustness of our results to the presence of competitors but our results did not qualitatively change. The results of this specification are reported as Table 11.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lambrecht, A., Tucker, C. Apparent algorithmic discrimination and real-time algorithmic learning in digital search advertising. Quant Mark Econ 22, 357–387 (2024). https://doi.org/10.1007/s11129-024-09286-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11129-024-09286-z