research-article

Model agnostic interpretability of rankers via intent modelling

Authors:

Jaspreet Singh,

Avishek AnandAuthors Info & Claims

FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

Pages 618 - 628

https://doi.org/10.1145/3351095.3375234

Published: 27 January 2020 Publication History

Abstract

A key problem in information retrieval is understanding the latent intention of a user's under-specified query. Retrieval models that are able to correctly uncover the query intent often perform well on the document ranking task. In this paper we study the problem of interpretability for text based ranking models by trying to unearth the query intent as understood by complex retrieval models.

We propose a model-agnostic approach that attempts to locally approximate a complex ranker by using a simple ranking model in the term space. Given a query and a blackbox ranking model, we propose an approach that systematically exploits preference pairs extracted from the target ranking and document perturbations to identify a set of intent terms and a simple term based ranker that can faithfully and accurately mimic the complex blackbox ranker in that locality. Our results indicate that we can indeed interpret more complex models with high fidelity. We also present a case study on how our approach can be used to interpret recently proposed neural rankers.

References

[1]

Nir Ailon, Moses Charikar, and Alantha Newman. 2008. Aggregating inconsistent information: ranking and clustering. Journal of the ACM (JACM) 55, 5 (2008), 23.

Digital Library

[2]

David Alvarez-Melis and Tommi S Jaakkola. 2017. A causal framework for explaining the predictions of black-box sequence-to-sequence models. arXiv preprint arXiv:1707.01943 (2017).

[3]

Leif Azzopardi, Wim Vanderbauwhede, and Hideo Joho. 2010. Search system requirements of patent analysts. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, 775--776.

Digital Library

[4]

Krisztian Balog, Filip Radlinski, and Shushan Arakelyan. 2019. Transparent, Scrutable and Explainable User Models for Personalized Recommendation. In Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). ACM, New York, NY, USA, 265--274.

Digital Library

[5]

Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek. 2016. Layer-wise relevance propagation for neural networks with local renormalization layers. In International Conference on Artificial Neural Networks. Springer, 63--71.

[6]

Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1721--1730.

Digital Library

[7]

Shuo Chang, F. Maxwell Harper, and Loren Gilbert Terveen. 2016. Crowd-Based Personalized Natural Language Explanations for Recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys '16). ACM, New York, NY, USA, 175--182.

Digital Library

[8]

Paul Alexandru Chirita, Rita Gavriloaie, Stefania Ghita, Wolfgang Nejdl, and Raluca Paiu. 2005. Activity based metadata for semantic desktop search. In European Semantic Web Conference. Springer, 439--454.

Digital Library

[9]

W Bruce Croft and John Lafferty. 2013. Language modeling for information retrieval. Vol. 13. Springer Science & Business Media.

[10]

Piotr Dabkowski and Yarin Gal. 2017. Real Time Image Saliency for Black Box Classifiers. arXiv preprint arXiv:1705.07857 (2017).

[11]

Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kamps, and W Bruce Croft. 2017. Neural Ranking Models with Weak Supervision. arXiv preprint arXiv:1704.08803 (2017).

[12]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[13]

Fernando Diaz, Bhaskar Mitra, and Nick Craswell. 2016. Query Expansion with Locally-Trained Word Embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 367--377.

[14]

Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. (2017).

[15]

Zeon Trevor Fernando, Jaspreet Singh, and Avishek Anand. 2019. A Study on the Interpretability of Neural Retrieval Models Using DeepSHAP. In Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). ACM, New York, NY, USA, 1005--1008.

Digital Library

[16]

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 51, 5 (2018), 93.

[17]

Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 55--64.

Digital Library

[18]

David Hawking. 2004. Challenges in enterprise search. In Proceedings of the 15th Australasian database conference-Volume 27. Australian Computer Society, Inc., 15--24.

Digital Library

[19]

Kai Hui, Andrew Yates, Klaus Berberich, and Gerard de Melo. 2017. A Position-Aware Deep Model for Relevance Matching in Information Retrieval. arXiv preprint arXiv:1704.03940 (2017).

[20]

Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Dan Roth. 2017. Learning what is essential in questions. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017). 80--89.

[21]

Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. 2016. Examples are not enough, learn to criticize! criticism for interpretability. In Advances in Neural Information Processing Systems. 2280--2288.

[22]

Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. arXiv preprint arXiv:1703.04730 (2017).

[23]

Victor Lavrenko and W. Bruce Croft. 2001. Relevance Based Language Models. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '01). ACM, New York, NY, USA, 120--127.

Digital Library

[24]

Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. arXiv preprint arXiv:1606.04155 (2016).

[25]

Benjamin Letham, Cynthia Rudin, Tyler H McCormick, David Madigan, et al. 2015. Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics 9, 3 (2015), 1350--1371.

[26]

Canjia Li, Yingfei Sun, Ben He, Le Wang, Kai Hui, Andrew Yates, Le Sun, and Jungang Xu. 2018. NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

[27]

Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2015. Visualizing and understanding neural models in nlp. arXiv preprint arXiv:1506.01066 (2015).

[28]

Tie-Yan Liu et al. 2009. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval 3, 3 (2009), 225--331.

[29]

Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765--4774.

[30]

Ryan McDonald, George Brokos, and Ion Androutsopoulos. 2018. Deep Relevance Ranking Using Enhanced Document-Query Interactions. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. ACL, 1849--1860. http://aclweb.org/anthology/D18-1211

[31]

Bhaskar Mitra and Nick Craswell. 2017. Neural Models for Information Retrieval. arXiv preprint arXiv:1705.01509 (2017).

[32]

Mandar Mitra, Amit Singhal, and Chris Buckley. 1998. Improving automatic query expansion. In SIGIR, Vol. 98. 206--214.

Digital Library

[33]

W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. 2019. Interpretable machine learning: definitions, methods, and applications. arXiv preprint arXiv:1901.04592 (2019).

[34]

Eric Nalisnick, Bhaskar Mitra, Nick Craswell, and Rich Caruana. 2016. Improving Document Ranking with Dual Word Embeddings. In Proceedings of the 25th International Conference Companion on World Wide Web (WWW '16 Companion). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 83--84.

Digital Library

[35]

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human-Generated MAchine Reading COmprehension Dataset. (2016).

[36]

Rodrigo Nogueira and Kyunghyun Cho. 2017. Task-oriented query reformulation with reinforcement learning. arXiv preprint arXiv:1704.04572 (2017).

[37]

Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and R Ward. 2014. Semantic modelling with long-short-term memory for information retrieval. arXiv preprint arXiv:1412.6629 (2014).

[38]

Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, and Xueqi Cheng. 2016. A Study of MatchPyramid Models on Ad-hoc Retrieval. SIGIR workshop on Neural Information Retrieval (NeuIR-16) arXiv:1606.04648 (2016). arXiv:1606.04648 http://arxiv.org/abs/1606.04648

[39]

Jay M Ponte and W Bruce Croft. 1998. A language modeling approach to information retrieval. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 275--281.

Digital Library

[40]

Yonggang Qiu and Hans-Peter Frei. 1993. Concept based query expansion. In Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 160--169.

Digital Library

[41]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. arXiv preprint arXiv:1606.05386 (2016).

[42]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why Should I Trust You?: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1135--1144.

Digital Library

[43]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In Thirty-Second AAAI Conference on Artificial Intelligence.

[44]

Stephen E Robertson. 1990. On term selection for query expansion. Journal of documentation 46, 4 (1990), 359--364.

Digital Library

[45]

Joseph Rocchio. 1971. Relevance feedback in information retrieval. The Smart retrieval system-experiments in automatic document processing (1971), 313--323.

[46]

Boris Sharchilev, Yury Ustinovskiy, Pavel Serdyukov, and Maarten Rijke. 2018. Finding Influential Training Samples for Gradient Boosted Decision Trees. In International Conference on Machine Learning. 4584--4592.

[47]

Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. 2014. Learning Semantic Representations Using Convolutional Neural Networks for Web Search. In Proceedings of the 23rd International Conference on World Wide Web (WWW '14 Companion). ACM, New York, NY, USA, 373--374.

Digital Library

[48]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013).

[49]

Jaspreet Singh and Avishek Anand. 2018. Posthoc Interpretability of Learning to Rank Models using Secondary Training Data. arXiv preprint arXiv:1806.11330 (2018).

[50]

Jaspreet Singh and Avishek Anand. 2019. EXS: Explainable Search Using Local Model Agnostic Interpretability. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (WSDM '19). ACM, New York, NY, USA, 770--773.

Digital Library

[51]

Maartje ter Hoeve, Anne Schuth, Daan Odijk, and Maarten de Rijke. 2018. Faithfully Explaining Rankings in a News Recommender System. arXiv preprint arXiv:1805.05447 (2018).

[52]

George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics 16, 1 (2015), 138.

[53]

Manisha Verma and Debasis Ganguly. 2019. LIRME: Locally Interpretable Ranking Model Explanation. In Proceedings of the 42Nd International ACM SIGIR.

Digital Library

[54]

Ellen M Voorhees. 1994. Query expansion using lexical-semantic relations. In SIGIR'94. Springer, 61--69.

Digital Library

[55]

Ellen M Voorhees, Donna K Harman, et al. 2005. TREC: Experiment and evaluation in information retrieval. Vol. 63. MIT press Cambridge.

Digital Library

[56]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In International Conference on Machine Learning. 2048--2057.

Digital Library

[57]

Yang Xu, Gareth JF Jones, and Bin Wang. 2009. Query dependent pseudo-relevance feedback based on wikipedia. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. ACM, 59--66.

Digital Library

[58]

Yongfeng Zhang, Yi Zhang, Min Zhang, and Chirag Shah. 2019. EARS 2019: The 2Nd International Workshop on ExplainAble Recommendation and Search. In Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). ACM, New York, NY, USA, 1438--1440.

Digital Library

Cited By

Xu ZLamba HAi QTetreault JJaimes AOosterhuis HBast HXiong C(2024)CFE2: Counterfactual Editing for Search Result ExplanationProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672508(145-155)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672508
Wallat JHinrichs HAnand ASerra ESpezzano F(2024)Causal Probing for Dual EncodersProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679556(2292-2303)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679556
Anand ASaha SSen PMitra M(2023)Explainability of Text Processing and Retrieval MethodsProceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation10.1145/3632754.3632944(153-157)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3632754.3632944
Show More Cited By

Model agnostic interpretability of rankers via intent modelling
1. Information systems

Recommendations

Characterizing commercial intent
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Understanding the intent underlying user's queries may help personalize search results and therefore improve user satisfaction. We develop a methodology for using the content of search engine result pages (SERPs) along with the information obtained from ...
Query intent inference via search engine log

Mining the latent intents behind search queries is critical for contemporary search engines. Therefore, there has been lots of effort on studying how to infer the intents of search queries via search engine query log. However, the task of query log-...
Intent-Aware Propensity Estimation via Click Pattern Stratification
WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023

Counterfactual learning to rank via inverse propensity weighting is the most popular approach to train ranking models using biased implicit user feedback from logged search data. Standard click propensity estimation techniques rely on simple models of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

January 2020

895 pages

ISBN:9781450369367

DOI:10.1145/3351095

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Amazon Research Award

Conference

FAT* '20

Sponsor:

ACM

FAT* '20: Conference on Fairness, Accountability, and Transparency

January 27 - 30, 2020

Barcelona, Spain

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
876
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)4

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu ZLamba HAi QTetreault JJaimes AOosterhuis HBast HXiong C(2024)CFE2: Counterfactual Editing for Search Result ExplanationProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672508(145-155)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672508
Wallat JHinrichs HAnand ASerra ESpezzano F(2024)Causal Probing for Dual EncodersProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679556(2292-2303)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679556
Anand ASaha SSen PMitra M(2023)Explainability of Text Processing and Retrieval MethodsProceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation10.1145/3632754.3632944(153-157)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3632754.3632944
Xu ZZeng HTan JFu ZZhang YAi Q(2023)A Reusable Model-agnostic Framework for Faithfully Explainable Recommendation and System ScrutabilityACM Transactions on Information Systems10.1145/360535742:1(1-29)Online publication date: 21-Aug-2023
https://dl.acm.org/doi/10.1145/3605357
Lucchese CMinello GNardini FOrlando SPerego RVeneri AFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Can Embeddings Analysis Explain Large Language Model Ranking?Proceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615225(4150-4154)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615225
Leonhardt JRudra KAnand A(2023)Extractive Explanations for Interpretable Text RankingACM Transactions on Information Systems10.1145/357692441:4(1-31)Online publication date: 23-Mar-2023
https://dl.acm.org/doi/10.1145/3576924
Anand ASen PSaha SVerma MMitra MChen HDuh WHuang HKato MMothe JPoblete B(2023)Explainable Information RetrievalProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3594249(3448-3451)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3594249
Lyu LAnand A(2023)Listwise Explanations for Ranking Models Using Multiple ExplainersAdvances in Information Retrieval10.1007/978-3-031-28244-7_41(653-668)Online publication date: 2-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-28244-7_41
Wallat JBeringer FAnand AAnand A(2023)Probing BERT for Ranking AbilitiesAdvances in Information Retrieval10.1007/978-3-031-28238-6_17(255-273)Online publication date: 2-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-28238-6_17
Wang YTanaka SYokoyama KWu HFang YCrestani FPasi GGaussier E(2022)Two-sided Rank Consistent Ordinal Regression for Interpretable Music Key RecommendationProceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3539813.3545147(223-231)Online publication date: 23-Aug-2022
https://dl.acm.org/doi/10.1145/3539813.3545147
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten