research-article

Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting

Authors:

Sheng-Chieh Lin,

Jheng-Hong Yang,

Rodrigo Nogueira,

Ming-Feng Tsai,

Jimmy LinAuthors Info & Claims

ACM Transactions on Information Systems (TOIS), Volume 39, Issue 4

Article No.: 48, Pages 1 - 29

https://doi.org/10.1145/3446426

Published: 01 September 2021 Publication History

Abstract

Conversational search plays a vital role in conversational information seeking. As queries in information seeking dialogues are ambiguous for traditional ad hoc information retrieval (IR) systems due to the coreference and omission resolution problems inherent in natural language dialogue, resolving these ambiguities is crucial. In this article, we tackle conversational passage retrieval, an important component of conversational search, by addressing query ambiguities with query reformulation integrated into a multi-stage ad hoc IR system. Specifically, we propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, stand-alone, human-understandable queries with a pretrained sequence-to-sequence model. Detailed analyses of the two CQR methods are provided quantitatively and qualitatively, explaining their advantages, disadvantages, and distinct behaviors. Moreover, to leverage the strengths of both CQR methods, we propose combining their output with reciprocal rank fusion, yielding state-of-the-art retrieval effectiveness, 30% improvement in terms of NDCG@3 compared to the best submission of Text REtrieval Conference (TREC) Conversational Assistant Track (CAsT) 2019.

References

[1]

Wasi Uddin Ahmad, Kai-Wei Chang, and Hongning Wang. 2018. Multi-task learning for document ranking and query suggestion. In Proc. ICLR.

[2]

Wasi Uddin Ahmad, Kai-Wei Chang, and Hongning Wang. 2019. Context attentive document ranking and query suggestion. In Proc. SIGIR. 385–394.

Digital Library

[3]

Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, and Jimmy Lin. 2019. Applying BERT to document retrieval with birch. In Proc. EMNLP: System Demonstrations. 19–24.

[4]

Mohammad Aliannejadi, Manajit Chakraborty, Esteban Andrés Ríssola, and Fabio Crestani. 2020. Harnessing evolution of multi-turn conversations for effective answer retrieval. In Proc. CHIIR. 33–42.

Digital Library

[5]

Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2019. Asking clarifying questions in open-domain information-seeking conversations. In Proc. SIGIR. 475–484.

Digital Library

[6]

Nima Asadi and Jimmy Lin. 2013. Document vector representations for feature extraction in multi-stage document ranking. J. Inf. Retr. 16, 6 (2013), 747–768.

Digital Library

[7]

Nima Asadi and Jimmy Lin. 2013. Effectiveness/efficiency tradeoffs for candidate generation in multi-stage retrieval architectures. In Proc. SIGIR. 997–1000.

Digital Library

[8]

Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. MS MARCO: A human generated machine reading comprehension dataset. Proc. of NIPS: the Workshop on Cognitive Computation. 1773.

[9]

Nicholas J. Belkin, C. Cool, W. Bruce Croft, and James P. Callan. 1993. The effect of multiple query representations on information retrieval system performance. In Proc. SIGIR. 339–346.

Digital Library

[10]

Rodger Benham, Joel Mackenzie, Alistair Moffat, and J. Shane Culpepper. 2019. Boosting search performance using query variations. ACM Trans. Inf. Syst. 37, 4, Article 41 (2019), 25 pages.

Digital Library

[11]

Anirban Chakraborty, Debasis Ganguly, Annalina Caputo, and Séamus Lawless. 2019. A factored relevance model for contextual point-of-interest recommendation. In Proc. ICTIR. 157–164.

Digital Library

[12]

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proc. EMNLP. 1724–1734.

[13]

Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: Question answering in context. arXiv:1808.07036. Retrieved from https://arxiv.org/abs/1808.07036.

[14]

Charles L. A. Clarke, J. Shane Culpepper, and Alistair Moffat. 2016. Assessing efficiency–effectiveness tradeoffs in multi-stage retrieval systems without using relevance judgments. J. Inf. Retr. 19, 4 (2016), 351–377.

Digital Library

[15]

Gordon V. Cormack, Charles L. A. Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proc. SIGIR. 758–759.

Digital Library

[16]

Stephen Cronen-Townsend, Yun Zhou, and W. Bruce Croft. 2002. Predicting query performance. In Proc. SIGIR. 299–306.

Digital Library

[17]

J. Shane Culpepper, Charles L. A. Clarke, and Jimmy Lin. 2016. Dynamic cutoff prediction in multi-stage retrieval systems. In Proc. ADCS. 17–24.

Digital Library

[18]

J. Shane Culpepper, Fernando Diaz, and Mark D. Smucker. 2018. Research frontiers in information retrieval: Report from the third strategic workshop on information retrieval in lorne. SIGIR Forum 52, 1 (2018), 34–90.

Digital Library

[19]

Zhuyun Dai and Jamie Callan. 2019. Deeper text understanding for IR with contextual neural language modeling. In Proc. SIGIR. 985–988.

Digital Library

[20]

Zhuyun Dai and Jamie Callan. 2020. Context-aware term weighting for first stage passage retrieval. In Proc. SIGIR. 1533–1536.

Digital Library

[21]

Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2019. CAsT 2019: The conversational assistance track overview. In Proc. TREC.

[22]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. NAACL. 4171–4186.

[23]

Laura Dietz and Nick Craswell. 2018. TREC complex answer retrieval overview. In Proc. TREC.

[24]

Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Proc. NIPS.

Digital Library

[25]

Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019. Can you unpack that? Learning to rewrite questions-in-context. In Proc. EMNLP. 5917–5923.

[26]

Luke Gallagher, Ruey-Cheng Chen, Roi Blanco, and J. Shane Culpepper. 2019. Joint optimization of cascade ranking models. In Proc. WSDM. 15–23.

Digital Library

[27]

Dan Hendrycks, Kimin Lee, and Mantas Mazeika. 2019. Using pre-training can improve model robustness and uncertainty. In Proc. ICML, Vol. 97. 2712–2721.

[28]

Vineet Kumar and Sachindra Joshi. 2017. Incomplete follow-up question resolution using retrieval based sequence to sequence learning. In Proc. SIGIR. 705–714.

Digital Library

[29]

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. Trans. ACL (2019).

[30]

Victor Lavrenko and W. Bruce Croft. 2001. Relevance based language models. In Proc. SIGIR. 120–127.

Digital Library

[31]

Nir Levine, Haggai Roitman, and Doron Cohen. 2017. An extended relevance model for session search. In Proc. SIGIR. 865–868.

Digital Library

[32]

Sheng-Chieh Lin, Jheng-Hong Yang, Rodrigo Nogueira, Ming-Feng Tsai, Chuan-Ju Wang, and Jimmy Lin. 2020. Conversational question reformulation via sequence-to-sequence architectures and pretrained language models. arXiv:2004.01909. Retrieved from https://arxiv.org/abs/2004.01909.

[33]

Yuanhua Lv and ChengXiang Zhai. 2009. A comparative study of methods for estimating query language models with pseudo feedback. In Proc. CIKM.

Digital Library

[34]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9 (2008), 2579–2605.

[35]

Hafeezul Rahman Mohammad, Keyang Xu, Jamie Callan, and J. Shane Culpepper. 2018. Dynamic shard cutoff prediction for selective search. In Proc. SIGIR. 85–94.

Digital Library

[36]

Rodrigo Nogueira and Kyunghyun Cho. 2017. Task-oriented query reformulation with reinforcement learning. In Proc. EMNLP. 574–583.

[37]

Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. arXiv:1901.04085. Retrieved from https://arxiv.org/abs/1901.04085.

[38]

Rodrigo Nogueira and Jimmy Lin. 2019. From doc2query to docTTTTTquery. Retrieved from https://cs.uwaterloo.ca/ jimmylin/publications/Nogueira_Lin_2019_docTTTTTquery-v2.pdf.

[39]

Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-stage document ranking with BERT. arXiv:1910.14424. Retrieved from https://arxiv.org/abs/1910.14424.

[40]

Terry Noreault, Michael McGill, and Matthew B. Koll. 1980. A performance evaluation of similarity measures, document term weighting schemes and representations in a boolean environment. In Proc. SIGIR. 57–76.

Digital Library

[41]

Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 10 (2010), 1345–1359.

Digital Library

[42]

Gustavo Penha, Alexandru Balan, and Claudia Hauff. 2019. Introducing MANtIS: A novel multi-domain information seeking dialogues dataset. arXiv:1912.04639. Retrieved from https://arxiv.org/abs/1912.04639.

[43]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2018. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2018), 9.

[44]

Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proc. CHIIR. 117–126.

Digital Library

[45]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 140 (2020), 1–67.

[46]

Gary Ren, Xiaochuan Ni, Manish Malik, and Qifa Ke. 2018. Conversational query understanding using sequence to sequence modeling. In Proc. WWW. 1715–1724.

Digital Library

[47]

Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3. In Proc. TREC.

[48]

Haggai Roitman. 2019. Normalized query commitment revisited. In Proc. SIGIR. 1085–1088.

Digital Library

[49]

Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nădejde. 2017. Nematus: A toolkit for neural machine translation. In Proc. EACL: Software Demonstrations. 65–68.

[50]

Anna Shtok, Oren Kurland, David Carmel, Fiana Raiber, and Gad Markovits. 2012. Predicting query performance by query-drift estimation. ACM Trans. Inf. Syst. 30, 2, Article 11 (2012), 35 pages.

Digital Library

[51]

Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. 2015. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In Proc. CIKM. 553–562.

Digital Library

[52]

Sandeep Subramanian, Raymond Li, Jonathan Pilault, and Christopher Pal. 2019. On extractive and abstractive neural document summarization with transformer language models. arXiv:1909.03186. Retrieved from https://arxiv.org/abs/1909.03186.

[53]

Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In Proc. EMNLP. 4231–4242.

[54]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proc. NIPS. 3104–3112.

Digital Library

[55]

Stephen Tomlinson. 2004. Robust, web and terabyte retrieval with hummingbird searchserver at TREC 2004. In Proc. TREC.

[56]

Nicola Tonellotto, Craig Macdonald, and Iadh Ounis. 2013. Efficient and effective retrieval using selective pruning. In Proc. WSDM. 63–72.

Digital Library

[57]

Zhucheng Tu, Matt Crane, Royal Sequiera, Junchen Zhang, and Jimmy Lin. 2017. An exploration of approaches to integrating neural reranking models in multi-stage ranking architectures. arXiv:1707.08275. Retrieved from https://arxiv.org/abs/1707.08275.

[58]

Svitlana Vakulenko, Shayne Longpre, Zhucheng Tu, and Raviteja Anantha. 2020. Question rewriting for conversational question answering. arXiv:2004.14652. Retrieved from https://arxiv.org/abs/2004.14652.

Digital Library

[59]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proc. NIPS. 5998–6008.

Digital Library

[60]

Ellen M. Voorhees. 2004. Overview of the TREC 2004 robust retrieval track. Proc. TREC.

[61]

Nikos Voskarides, Dan Li, Andreas Panteli, and Pengjie Ren. 2019. ILPS at TREC 2019 conversational assistant track. In Proc. TREC.

[62]

Nikos Voskarides, Dan Li, Pengjie Ren, Evangelos Kanoulas, and Maarten de Rijke. 2020. Query resolution for conversational search with limited supervision. In Proc. SIGIR.

Digital Library

[63]

Shuohang Wang, Mo Yu, Jing Jiang, Wei Zhang, Xiaoxiao Guo, Shiyu Chang, Zhiguo Wang, Tim Klinger, Gerald Tesauro, and Murray Campbell. 2018. Evidence aggregation for answer re-ranking in open-domain question answering. In Proc. ICLR.

[64]

J. Xu and W. B. Croft. 1996. Query expansion using local and global document analysis. In Proc. SIGIR. 4–11.

Digital Library

[65]

Hui Yang, Dongyi Guan, and Sicong Zhang. 2015. The query change model: Modeling session search as a markov decision process. ACM Trans. Inf. Syst. 33, 4, Article 20 (2015), 33 pages.

Digital Library

[66]

Jheng-Hong Yang, Sheng-Chieh Lin, Chuan-Ju Wang, Jimmy Lin, and Ming-Feng Tsai. 2019. Query and answer expansion from conversation history. In Proc. TREC.

[67]

Peilin Yang, Hui Fang, and Jimmy Lin. 2018. Anserini: Reproducible ranking baselines using lucene. J. Data Inf. Qual. 10, 4 (2018), 16.

Digital Library

[68]

Shi Yu, Jiahua Liu, Jingqin Yang, Chenyan Xiong, Paul Bennett, Jianfeng Gao, and Zhiyuan Liu. 2020. Few-shot generative conversational query rewriting. In Proc. SIGIR. 1933–1936.

Digital Library

[69]

Oleg Zendel, Anna Shtok, Fiana Raiber, Oren Kurland, and J. Shane Culpepper. 2019. Information needs, queries, and query performance prediction. In Proc. SIGIR. 395–404.

Digital Library

[70]

Yun Zhou and W. Bruce Croft. 2007. Query performance prediction in web search environments. In Proc. SIGIR. 543–550.

Digital Library

[71]

Shihao Zou, Guanyu Tao, Jun Wang, Weinan Zhang, and Dell Zhang. 2018. On the equilibrium of query reformulation and document retrieval. In Proc. SIGIR. 43–50.

Digital Library

Cited By

Ye ZXie XAi QLiu YWang ZSu WZhang M(2024)Relevance Feedback with Brain SignalsACM Transactions on Information Systems10.1145/363787442:4(1-37)Online publication date: 9-Feb-2024
https://dl.acm.org/doi/10.1145/3637874
Kostric IBalog KHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)A Surprisingly Simple yet Effective Multi-Query Rewriting Method for Conversational Passage RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657933(2271-2275)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657933
Mo FYi BMao KQu CHuang KNie JChua TNgo CKumar RLauw HKa-Wei Lee R(2024)ConvSDG: Session Data Generation for Conversational SearchCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651940(1634-1642)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651940
Show More Cited By

Index Terms

Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting
1. Information systems
  1. Information retrieval

Recommendations

Query Resolution for Conversational Search with Limited Supervision
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

In this work we focus on multi-turn passage retrieval as a crucial component of conversational search. One of the key challenges in multi-turn passage retrieval comes from the fact that the current turn query is often underspecified due to zero anaphora,...
Few-Shot Generative Conversational Query Rewriting
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Conversational query rewriting aims to reformulate a concise conversational query to a fully specified, context-independent query that can be effectively handled by existing information retrieval systems. This paper presents a few-shot generative ...
Question Rewriting for Conversational Question Answering
WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Conversational question answering (QA) requires the ability to correctly interpret a question in the context of previous conversation turns. We address the conversational QA task by decomposing it into question rewriting and question answering subtasks. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems

ACM Transactions on Information Systems Volume 39, Issue 4

October 2021

482 pages

ISSN:1046-8188

EISSN:1558-2868

DOI:10.1145/3477247

Editor:
Min Zhang
Tsinghua University, China

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2021

Accepted: 01 December 2020

Revised: 01 December 2020

Received: 01 May 2020

Published in TOIS Volume 39, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

Canada First Research Excellence Fund
Natural Sciences and Engineering Research Council (NSERC)
Ministry of Science and Technology in Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
403
Total Downloads

Downloads (Last 12 months)76
Downloads (Last 6 weeks)19

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ye ZXie XAi QLiu YWang ZSu WZhang M(2024)Relevance Feedback with Brain SignalsACM Transactions on Information Systems10.1145/363787442:4(1-37)Online publication date: 9-Feb-2024
https://dl.acm.org/doi/10.1145/3637874
Kostric IBalog KHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)A Surprisingly Simple yet Effective Multi-Query Rewriting Method for Conversational Passage RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657933(2271-2275)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657933
Mo FYi BMao KQu CHuang KNie JChua TNgo CKumar RLauw HKa-Wei Lee R(2024)ConvSDG: Session Data Generation for Conversational SearchCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651940(1634-1642)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651940
Ahmed M(2023)Automatic Indexing for Agriculture: Designing a Framework by Deploying Agrovoc, Agris and AnnifJournal of Information and Knowledge10.17821/srels/2023/v60i2/170966(85-95)Online publication date: 13-May-2023
https://doi.org/10.17821/srels/2023/v60i2/170966
Bernard NBalog K(2023)A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information RetrievalACM Computing Surveys10.1145/3637211Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3637211
Krasakis AYates AKanoulas E(2023)Contextualizing and Expanding Conversational Queries without SupervisionACM Transactions on Information Systems10.1145/363262242:3(1-30)Online publication date: 30-Dec-2023
https://dl.acm.org/doi/10.1145/3632622
Acharya PFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Towards Effective Modeling and Exploitation of Search and User Context in Conversational Information RetrievalProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3616005(5161-5164)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3616005
Yang DZhang YFang HYoshioka MKiseleva JAliannejadi M(2023)Zero-shot Query Reformulation for Conversational SearchProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605143(257-263)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605143
Mao KQian HMo FDou ZLiu BCheng XCao Z(2023)Learning Denoised and Interpretable Session Representation for Conversational SearchProceedings of the ACM Web Conference 202310.1145/3543507.3583265(3193-3202)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583265
Ju JLin STsai MWang CChen HDuh WHuang HKato MMothe JPoblete B(2023)Improving Conversational Passage Re-ranking with View EnsembleProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592002(2077-2081)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592002
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents