research-article

MOBIUS: Towards the Next Generation of Query-Ad Matching in Baidu's Sponsored Search

Authors:

Ping LiAuthors Info & Claims

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 2509 - 2517

https://doi.org/10.1145/3292500.3330651

Published: 25 July 2019 Publication History

Abstract

Baidu runs the largest commercial web search engine in China, serving hundreds of millions of online users every day in response to a great variety of queries. In order to build a high-efficiency sponsored search engine, we used to adopt a three-layer funnel-shaped structure to screen and sort hundreds of ads from billions of ad candidates subject to the requirement of low response latency and the restraints of computing resources. Given a user query, the top matching layer is responsible for providing semantically relevant ad candidates to the next layer, while the ranking layer at the bottom concerns more about business indicators (e.g., CPM, ROI, etc.) of those ads. The clear separation between the matching and ranking objectives results in a lower commercial return. The Mobius project has been established to address this serious issue. It is our first attempt to train the matching layer to consider CPM as an additional optimization objective besides the query-ad relevance, via directly predicting CTR (click-through rate) from billions of query-ad pairs. Specifically, this paper will elaborate on how we adopt active learning to overcome the insufficiency of click history at the matching layer when training our neural click networks offline, and how we use the SOTA ANN search technique for retrieving ads more efficiently (Here "ANN'' stands for approximate nearest neighbor search). We contribute the solutions to Mobius-V1 as the first version of our next generation query-ad matching system.

References

[1]

Vibhanshu Abhishek and Kartik Hosanagar. 2007. Keyword Generation for Search Engine Advertising Using Semantic Similarity between Terms. In Proceedings of the 9th International Conference on Electronic Commerce (EC). Minneapolis, MN, 89--94.

Digital Library

[2]

Ricardo Baeza-Yates, Massimiliano Ciaramita, Peter Mika, and Hugo Zaragoza. 2008. Towards Semantic Search. In International Conference on Application of Natural Language to Information Systems. Springer, 4--11.

Digital Library

[3]

Xiao Bai, Erik Ordentlich, Yuanyuan Zhang, Andy Feng, Adwait Ratnaparkhi, Reena Somvanshi, and Aldi Tjahjadi. 2018. Scalable Query N-Gram Embedding for Improving Matching and Relevance in Sponsored Search. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). London, UK, 52--61.

Digital Library

[4]

Andrei Broder, Peter Ciccolo, Evgeniy Gabrilovich, Vanja Josifovski, Donald Metzler, Lance Riedel, and Jeffrey Yuan. 2009. Online Expansion of Rare Queries for Sponsored Search. In Proceedings of the 18th International conference on World Wide Web (WWW). Madrid, Spain, 511--520.

Digital Library

[5]

Andrei Z. Broder, Steven C. Glassman, Mark S. Manasse, and Geoffrey Zweig. 1997. Syntactic Clustering of the Web. Computer Networks, Vol. 29, 8--13 (1997), 1157--1166.

Digital Library

[6]

Patrick P. K. Chan, Xian Hu, Lili Zhao, Daniel S. Yeung, Dapeng Liu, and Lei Xiao. 2018. Convolutional Neural Networks based Click-Through Rate Prediction with Multiple Feature Sequences. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI). Stockholm, Sweden, 2007--2013.

Digital Library

[7]

Moses S Charikar. 2002. Similarity Estimation Techniques from Rounding Algorithms. In Proceedings on 34th Annual ACM Symposium on Theory of Computing (STOC). Montré al, Qué bec, Canada, 380--388.

Digital Library

[8]

Sanjoy Dasgupta and Yoav Freund. 2008. Random Projection Trees and Low Dimensional Manifolds. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC). Victoria, British Columbia, Canada, 537--546.

Digital Library

[9]

Sanjoy Dasgupta and Kaushik Sinha. 2015. Randomized Partition Trees for Nearest Neighbor Search. Algorithmica, Vol. 72, 1 (2015), 237--263.

Digital Library

[10]

Kushal Dave and Vasudeva Varma. 2014. Computational Advertising: Techniques for Targeting Relevant Ads. Foundations and Trends in Information Retrieval, Vol. 8 (Oct. 2014), 263--418.

Digital Library

[11]

Daniel C Fain and Jan O Pedersen. 2006. Sponsored Search: A Brief History. Bulletin of the American Society for Information Science and Technology, Vol. 32, 2 (2006), 12--13.

[12]

Jerome H. Friedman, F. Baskett, and L. Shustek. 1975. An Algorithm for Finding Nearest Neighbors. IEEE Trans. Comput., Vol. 24 (1975), 1000--1006.

Digital Library

[13]

Jerome H. Friedman, J. Bentley, and R. Finkel. 1977. An Algorithm for Finding Best Matches in Logarithmic Expected Time. ACM Trans. Math. Software, Vol. 3 (1977), 209--226.

Digital Library

[14]

Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized Product Quantization for Approximate Nearest Neighbor Search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2946--2953.

Digital Library

[15]

Aristides Gionis, Piotr Indyk, and Rajeev Motwani. 1999. Similarity Search in High Dimensions via Hashing. In Proceedings of 25th International Conference on Very Large Data Bases (VLDB). Edinburgh, Scotland, UK, 518--529.

Digital Library

[16]

Thore Graepel, Joaquin Qui n onero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-Scale Bayesian Click-Through Rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine. In Proceedings of the 27th International Conference on Machine Learning (ICML). 13--20.

Digital Library

[17]

Mihajlo Grbovic, Nemanja Djuric, Vladan Radosavljevic, Fabrizio Silvestri, Ricardo Baeza-Yates, Andrew Feng, Erik Ordentlich, Lee Yang, and Gavin Owens. 2016. Scalable Semantic Matching of Queries to Ads in Sponsored Search Advertising. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). Pisa, Italy, 375--384.

Digital Library

[18]

Mihajlo Grbovic, Nemanja Djuric, Vladan Radosavljevic, Fabrizio Silvestri, and Narayan Bhamidipati. 2015. Context- and Content-aware Embeddings for Query Rewriting in Sponsored Search. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). Santiago, Chile, 383--392.

Digital Library

[19]

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et almbox. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising (ADKDD). New York, NY, 1--9.

Digital Library

[20]

Piotr Indyk and Rajeev Motwani. 1998. Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality. In Proceedings of the Thirtieth Annual ACM Symposium on the Theory of Computing (STOC). Dallas, TX, 604--613.

Digital Library

[21]

Michael Jahrer, A Toscher, Jeong-Yoon Lee, J Deng, Hang Zhang, and Jacob Spoelstra. 2012. Ensemble of Collaborative Filtering and Feature Engineered Models for Click Through Rate Prediction. In KDDCup Workshop.

[22]

Bernard J Jansen and Tracy Mullen. 2008. Sponsored Search: An Overview of the Concept, History, and Technology. International Journal of Electronic Business, Vol. 6, 2 (2008), 114--131.

[23]

Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product Quantization for Nearest Neighbor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 33, 1 (2011), 117--128.

Digital Library

[24]

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale Similarity Search with GPUs. arXiv preprint arXiv:1702.08734 (2017).

[25]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep Learning. Nature, Vol. 521, 7553 (2015), 436.

[26]

Victor Lempitsky. 2012. The Inverted Multi-index. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, 3069--3076.

Digital Library

[27]

Ping Li, Art B Owen, and Cun-Hui Zhang. 2012a. One Permutation Hashing. In Advances in Neural Information Processing Systems (NIPS). Lake Tahoe, NV, 3122--3130.

Digital Library

[28]

Ping Li, Gennady Samorodnitsky, and John Hopcroft. 2013. Sign Cauchy Projections and Chi-Square Kernel. In Advances in Neural Information Processing Systems (NIPS). Lake Tahoe, NV, 2571--2579.

Digital Library

[29]

Ping Li, Anshumali Shrivastava, and Christian A. Konig. 2012b. GPU-based Minwise Hashing: GPU-based Minwise Hashing. In Proceedings of the 21st World Wide Web Conference (WWW). Lyon, France, 565--566.

Digital Library

[30]

Ping Li and Martin Slawski. 2017. Simple Strategies for Recovering Inner Products from Coarsely Quantized Random Projections. In Advances in Neural Information Processing Systems (NIPS). Long Beach, CA, USA, 4570--4579.

Digital Library

[31]

H. Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, and Jeremy Kubica. 2013. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). Chicago, IL, 1222--1230.

Digital Library

[32]

Hema Raghavan and Rukmini Iyer. 2008. Evaluating Vector-space and Probabilistic Models for Query to Ad Matching. In SIGIR Workshop on Information Retrieval in Advertising (IRA).

[33]

Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting Clicks: Estimating the Click-through Rate for New Ads. In Proceedings of the 16th International Conference on World Wide Web (WWW). Banff, Alberta, Canada, 521--530.

Digital Library

[34]

Burr Settles. 2012. Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 6, 1 (2012), 1--114.

Digital Library

[35]

Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. 2014. Learning Semantic Representations Using Convolutional Neural Networks for Web Search. In Proceedings of the 23rd International Conference on World Wide Web (WWW). Seoul, Korea, 373--374.

Digital Library

[36]

Anshumali Shrivastava and Ping Li. 2014a. Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS). In Advances in Neural Information Processing Systems (NIPS). Montré al, Qué bec, Canada, 2321--2329.

Digital Library

[37]

Anshumali Shrivastava and Ping Li. 2014b. In Defense of MinHash Over SimHash. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics (AISTATS). Reykjavik, Iceland, 886--894.

[38]

Shulong Tan, Zhixin Zhou, Zhaozhuo Xu, and Ping Li. 2019. Fast Item Ranking under Neural Network based Measures. Technical Report. Baidu Research.

[39]

Looja Tuladhar and Manish Satyapal Gupta. 2014. Click Through Rate Prediction System and Method. US Patent 8,738,436.

[40]

Haofen Wang, Yan Liang, Linyun Fu, Gui-Rong Xue, and Yong Yu. 2009. Efficient Query Expansion for Advertisement Search. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). Boston, MA, 51--58.

Digital Library

[41]

Meng Wang and Xian-Sheng Hua. 2011. Active Learning in Multimedia Annotation and Retrieval: A Survey. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 2, 2 (2011), 10.

Digital Library

[42]

Xiang Wu, Ruiqi Guo, Ananda Theertha Suresh, Sanjiv Kumar, Daniel N Holtmann-Rice, David Simcha, and Felix Yu. 2017. Multiscale Quantization for Fast Similarity Search. In Advances in Neural Information Processing Systems (NIPS). Long Beach, CA, 5745--5755.

Digital Library

[43]

Xiao Yan, Jinfeng Li, Xinyan Dai, Hongzhi Chen, and James Cheng. 2018. Norm-Ranging LSH for Maximum Inner Product Search. In Advances in Neural Information Processing Systems (NeurIPS). 2956--2965.

Digital Library

[44]

Hsiang-Fu Yu, Cho-Jui Hsieh, Qi Lei, and Inderjit S. Dhillon. 2017. A Greedy Approach for Budgeted Maximum Inner Product Search. In Advances in Neural Information Processing Systems (NIPS). Long Beach, CA, 5459--5468.

Digital Library

[45]

Wei Vivian Zhang, Xiaofei He, Benjamin Rey, and Rosie Jones. 2007. Query Rewriting Using Active Learning for Sponsored Search. In Proceedings of the 30th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). Amsterdam, The Netherlands, 853--854.

Digital Library

[46]

Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI). Qué bec City, Qué bec, Canada, 1369--1375.

Digital Library

[47]

Weijie Zhao, Shulong Tan, and Ping Li. 2019. SONG: Approximate Nearest Neighbor Search on GPU. Technical Report. Baidu Research.

[48]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-through Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). London, UK, 1059--1068.

Digital Library

Cited By

Lin JLi QXie GGuan ZJiang YXu TZhang ZZhao PCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Mitigating Sample Selection Bias with Robust Domain Adaption in Multimedia RecommendationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680615(7581-7590)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680615
Li PZhao WOosterhuis HBast HXiong C(2024)Pb-Hash: Partitioned b-bit HashingProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672523(239-246)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672523
Zhao WTan SLi PHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)GUITAR: Gradient Pruning toward Fast Neural RankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657728(163-173)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657728
Show More Cited By

Index Terms

MOBIUS: Towards the Next Generation of Query-Ad Matching in Baidu's Sponsored Search
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. World Wide Web
    1. Online advertising
      1. Sponsored search advertising
    2. Web searching and information discovery
      1. Web search engines
        Web indexing

Recommendations

Scalable Query N-Gram Embedding for Improving Matching and Relevance in Sponsored Search
KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Sponsored search has been the major source of revenue for commercial web search engines. It is crucial for a sponsored search engine to retrieve ads that are relevant to user queries to attract clicks as advertisers only pay when their ads get clicked. ...
The comparative effectiveness of sponsored and nonsponsored links for Web e-commerce queries

The predominant business model for Web search engines is sponsored search, which generates billions in yearly revenue. But are sponsored links providing online consumers with relevant choices for products and services? We address this and related issues ...
Query rewriting using active learning for sponsored search
SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

Sponsored search is a major revenue source for search companies. Web searchers can issue any queries, while advertisement keywords are limited. Query rewriting technique effectively matches user queries with relevant advertisement keywords, thus ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

July 2019

3305 pages

ISBN:9781450362016

DOI:10.1145/3292500

General Chairs:
Ankur Teredesai
KenSci
,
Vipin Kumar
University of Minnesota
,
Program Chairs:
Ying Li
EV Analysis Corporation
,
Rómer Rosales
LinkedIn
,
Evimaria Terzi
Boston University
,
George Karypis
University of Minnesota

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '19

Sponsor:

KDD '19: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 4 - 8, 2019

AK, Anchorage, USA

Acceptance Rates

KDD '19 Paper Acceptance Rate 110 of 1,200 submissions, 9%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

60
Total Citations
View Citations
1,678
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lin JLi QXie GGuan ZJiang YXu TZhang ZZhao PCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Mitigating Sample Selection Bias with Robust Domain Adaption in Multimedia RecommendationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680615(7581-7590)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680615
Li PZhao WOosterhuis HBast HXiong C(2024)Pb-Hash: Partitioned b-bit HashingProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672523(239-246)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672523
Zhao WTan SLi PHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)GUITAR: Gradient Pruning toward Fast Neural RankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657728(163-173)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657728
You XYang HWang SPeng TDing CLi XChen BLuan ZLiu TLi YQian D(2024)Exploiting Structured Feature and Runtime Isolation for High-Performant Recommendation ServingIEEE Transactions on Computers10.1109/TC.2024.344974973:11(2474-2487)Online publication date: Nov-2024
https://doi.org/10.1109/TC.2024.3449749
Wang QLi HXiong HWang WBian JLu YWang SCheng ZDou DYin D(2024)A Simple yet Effective Framework for Active Learning to RankMachine Intelligence Research10.1007/s11633-023-1422-z21:1(169-183)Online publication date: 15-Jan-2024
https://doi.org/10.1007/s11633-023-1422-z
Huang HBai YLiang HLiu X(2024)IR Embedding Fairness Inspection via Contrastive Learning and Human-AI Collaborative IntelligenceAdvances in Knowledge Discovery and Data Mining10.1007/978-981-97-2238-9_11(142-153)Online publication date: 1-May-2024
https://doi.org/10.1007/978-981-97-2238-9_11
Chen WZhang YXian YWen Y(2023)Hotspot Information Network and Domain Knowledge Graph Aggregation in Heterogeneous Network for Literature RecommendationApplied Sciences10.3390/app1302109313:2(1093)Online publication date: 13-Jan-2023
https://doi.org/10.3390/app13021093
Ni HWang Z(2023)Feature Dual Supervision Model for the Searches of Online Advertising AudiencesScientific Programming10.1155/2023/12178982023Online publication date: 11-May-2023
https://dl.acm.org/doi/10.1155/2023/1217898
Jiao XLi WWu XHu WLi MBian JDai SLuo XHu MHuang ZFeng DYang JFeng SXiong HYu DLi SHe JMa YLiu LSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)PGLBox: Multi-GPU Graph Learning Framework for Web-Scale RecommendationProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599885(4262-4272)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599885
Wang DSalamatian KXia YDeng WZhang QSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)BERT4CTR: An Efficient Framework to Combine Pre-trained Language Model with Non-textual Features for CTR PredictionProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599780(5039-5050)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599780
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten