Report on the 1st Workshop on Generative Information Retrieval (Gen-IR 2023) at SIGIR 2023
Authors: 
Gabriel Bénédict, Ruqing Zhang, 
Donald Metzler, Andrew Yates, Romain Deffayet, Philipp Hager, Sami JullienAuthors Info & Claims
Gabriel Bénédict, Ruqing Zhang, Donald Metzler, Andrew Yates, Romain Deffayet, Philipp Hager, Sami JullienAuthors Info & ClaimsArticle No.: 13, Pages 1 - 23
Abstract
The first edition of the workshop on Generative Information Retrieval (Gen-IR 2023) took place in July 2023 in a hybrid fashion, co-located with the ACM SIGIR Conference 2023 in Taipei (SIGIR 2023). The aim was to bring information retrieval researchers together around the topic of generative AI that gathered attention in 2022 and 2023 with large language models and diffusion models. Given the novelty of the topic, the workshop was focused around multi-sided discussions, namely panels and poster sessions of the accepted proceedings papers. Two main research outcomes are the proceedings of the workshop1 and the potential research directions discussed in this report.
Date: 27 July 2023.
Website: https://coda.io/@sigir/gen-ir.
References
[1]
Amin Abolghasemi, Suzan Verberne, Arian Askari, and Leif Azzopardi. Retrievability bias estimation using synthetically generated queries. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[2]
Arian Askari, Mohammad Aliannejadi, Evangelos Kanoulas, and Suzan Verberne. Generating synthetic documents for cross-encoder re-rankers: A comparative study of chatgpt and human experts. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[3]
Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Max Kaufmann, Meg Tong, Tomasz Korbak, Daniel Kokotajlo, and Owain Evans. Taken out of context: On measuring situational awareness in llms, 2023.
[4]
Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Scott Yih, Sebastian Riedel, and Fabio Petroni. Autoregressive search engines: Generating substrings as document identifiers. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 31668--31683. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/cd88d62a2063fdaf7ce6f9068fb15dcd-Paper-Conference.pdf.
[5]
Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. Autoregressive entity retrieval. CoRR, abs/2010.00904, 2020. URL https://arxiv.org/abs/2010.00904.
[6]
Shang Gao, Divyanshu Murli, Javed Qadrud-Din, and Martin Gajek. Gpt-4 synthetic data improves generalizability for contract clause retrieval. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[7]
Hado Hasselt. Double q-learning. Advances in neural information processing systems, 23, 2010.
[8]
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2790--2799. PMLR, 09--15 Jun 2019.
[9]
Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. Query expansion by prompting large language models. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[10]
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535--547, 2019.
[11]
Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 39--48, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380164.
[12]
Germán Kruszewski, Jos Rozen, and Marc Dymetman. disco: a toolkit for distributional control of generative models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 144--160, Toronto, Canada, July 2023. Association for Computational Linguistics.
[13]
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: a benchmark for question answering research. Transactions of the Association of Computational Linguistics, 2019.
[14]
Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Wei Chen, and Xueqi Cheng. On the robustness of generative retrieval models: An out-of-distribution perspective. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[15]
Pei-Chi Lo, Yi-Hang Tsai, Ee-Peng Lim, and San-Yih Hwang. On exploring the reasoning capability of large language models with knowledge graphs. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[16]
Thong Nguyen and Andrew Yates. Generative retrieval as dense retrieval. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[17]
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human generated machine reading comprehension dataset. In CoCo@NIPs, 2016.
[18]
Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864--1874. Association for Computational Linguistics, May 2022.
[19]
Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. Document ranking with a pretrained sequence-to-sequence model. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 708--718, Online, November 2020. Association for Computational Linguistics.
[20]
Aleksandr V Petrov and Craig Macdonald. Generative sequential recommendation with gptrec. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[21]
Ronak Pradeep, Kai Hui, Jai Gupta, Adam D Lelkes, Honglei Zhuang, Jimmy Lin, Donald Metzler, and Vinh Q Tran. How does generative retrieval scale to millions of passages? Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[22]
Stephen Robertson, S. Walker, M. M. Hancock-Beaulieu, M. Gatford, and A. Payne. Okapi at trec-4. In The Fourth Text REtrieval Conference (TREC-4), pages 73--96. Gaithersburg, MD: NIST, January 1996. URL https://www.microsoft.com/en-us/research/publication/okapi-at-trec-4/.
[23]
Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
[24]
Sajad Sotudeh and Nazli Goharian. Qontsum: On contrasting salient content for query-focused summarization. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[25]
Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. Transformer memory as a differentiable search index. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=Vu-B0clPfq.
[26]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023.
[27]
Xiao Wang, Sean MacAvaney, Craig Macdonald, and Iadh Ounis. Generative query reformulation for effective adhoc search. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[28]
Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, and Yanbin Lu. Palr: Personalization aware llms for recommendation. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[29]
Weijia Zhang, Svitlana Vakulenko, Thilina Rajapakse, Yumo Xu, and Evangelos Kanoulas. Tackling query-focused summarization as a knowledge-intensive task: A pilot study. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
[30]
Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon, and Daxin Jiang. Bridging the gap between indexing and retrieval for differentiable search index with query generation. Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval, 2023.
Index Terms
- Report on the 1st Workshop on Generative Information Retrieval (Gen-IR 2023) at SIGIR 2023
Index terms have been assigned to the content through auto-classification.
Recommendations
Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
The SIGIR 2016 workshop on Neural Information Retrieval (Neu-IR) took place on 21 July, 2016 in Pisa. The goal of the Neu-IR (pronounced "New IR") workshop was to serve as a forum for academic and industrial researchers, working at the intersection of ...
Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR'17)
The second SIGIR workshop on neural information retrieval (Neu-IR?17) took place on August 11, 2017, in Tokyo, Japan. Following the successful 2016 edition, the workshop continued to serve as a forum for academic and industrial researchers to present ...
ACM WSDM 2023 Report
The 16th ACM International Conference on Web Search and Data Mining (WSDM 2023) was successfully held on 2/27 - 3/3/2023 at Carlton Hotel, Singapore. It was held as in-person conference that also featured rich virtual elements. The five-day event began ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Copyright is held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 January 2024
Published in SIGIR Volume 57, Issue 2
Check for updates
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 120Total Downloads
- Downloads (Last 12 months)120
- Downloads (Last 6 weeks)10
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in