Propensity-Dependent Model for Unbiased Learning-to-Rank
Pages 1 - 4
Abstract
Most present unbiased learning-to-rank models are based on the trust bias assumption to learn a ranking policy by Inverse Propensity Scoring (IPS). The trust bias assumption improves the unrealistic noise-free assumption in the Position-Based model, but it assumes that the propensities are independent. In this paper, we improve this assumption and consider that the propensities of different positions are relevant. In particular, we model the relationship between different propensities as a Propensity-Dependent model and use both IPS estimator and the Doubly Robust estimator to learn the optimal ranking policy. Finally, we generate the dataset in a simulated study and then evaluate the model's performance.
References
[1]
Melville, Prem, and Vikas Sindhwani. "Recommender systems." Encyclopedia of machine learning 1 (2010): 829-838.
[2]
Chapelle, Olivier, and Yi Chang. "Yahoo! learning to rank challenge overview." Proceedings of the learning to rank challenge. PMLR, 2011.
[3]
Joachims, Thorsten. "Optimizing search engines using clickthrough data." Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. 2002.
[4]
Oosterhuis, Harrie. "Doubly-Robust Estimation for Unbiased Learning-to-Rank from Position-Biased Click Feedback." arXiv preprint arXiv:2203.17118 (2022).
[5]
Agarwal, Aman, "Addressing trust bias for unbiased learning-to-rank." The World Wide Web Conference. 2019.
[6]
Ai, Qingyao, "Unbiased learning to rank with unbiased propensity estimation." The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018.
[7]
Joachims, Thorsten, Adith Swaminathan, and Tobias Schnabel. "Unbiased learning-to-rank with biased feedback." Proceedings of the tenth ACM international conference on web search and data mining. 2017.
[8]
Kang, Joseph DY, and Joseph L. Schafer. "Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data." Statistical science 22.4 (2007): 523-539.
[9]
Vardasbi, Ali, Harrie Oosterhuis, and Maarten de Rijke. "When inverse propensity scoring does not work: Affine corrections for unbiased learning to rank." Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020.
[10]
Chapelle, Olivier, and Yi Chang. "Yahoo! learning to rank challenge overview." Proceedings of the learning to rank challenge. PMLR, 2011.
[11]
Qin, Tao, and Tie-Yan Liu. "Introducing LETOR 4.0 datasets." arXiv preprint arXiv: 1306. 2597 (2013).
[12]
Dato, Domenico, "Fast ranking with additive ensembles of oblivious and non-oblivious regression trees." ACM Transactions on Information Systems (TOIS) 35.2 (2016): 1-31.
[13]
Saito, Yuta. "Doubly robust estimator for ranking metrics with post-click conversions." Fourteenth ACM Conference on Recommender Systems. 2020.
[14]
Wang, Nan, "non-clicks mean irrelevant? propensity ratio scoring as a correction." Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 2021.
[15]
Saito, Yuta, "Unbiased recommender learning from missing-not-at-random implicit feedback." Proceedings of the 13th International Conference on Web Search and Data Mining. 2020.
[16]
Bekker, Jessa, Pieter Robberechts, and Jesse Davis. "Beyond the selected completely at random assumption for learning from positive and unlabeled data." Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Cham, 2019.
[17]
Wang, Xuanhui, "Position bias estimation for unbiased learning to rank in personal search." Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 2018.
Index Terms
- Propensity-Dependent Model for Unbiased Learning-to-Rank
Recommendations
Model-based Unbiased Learning to Rank
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data MiningUnbiased Learning to Rank(ULTR), i.e., learning to rank documents with biased user feedback data, is a well-known challenge in information retrieval. Existing methods in unbiased learning to rank typically rely on click modeling or inverse propensity ...
Position Bias Estimation for Unbiased Learning to Rank in Personal Search
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data MiningA well-known challenge in learning from click data is its inherent bias and most notably position bias. Traditional click models aim to extract the ‹query, document› relevance and the estimated bias is usually discarded after relevance is extracted. In ...
Unbiased Learning to Rank with Unbiased Propensity Estimation
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalLearning to rank with biased click data is a well-known challenge. A variety of methods has been explored to debias click data for learning to rank such as click models, result interleaving and, more recently, the unbiased learning-to-rank framework ...
Comments
Information & Contributors
Information
Published In
November 2022
87 pages
ISBN:9781450397940
DOI:10.1145/3582084
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 April 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICSED 2022
ICSED 2022: 2022 4th International Conference on Software Engineering and Development
November 25 - 27, 2022
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 33Total Downloads
- Downloads (Last 12 months)16
- Downloads (Last 6 weeks)0
Reflects downloads up to 16 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format