Article

Enhance Performance of Ad-hoc Search via Prompt Learning

Authors:

Shaoping MaAuthors Info & Claims

Information Retrieval: 28th China Conference, CCIR 2022, Chongqing, China, September 16–18, 2022, Revised Selected Papers

Pages 28 - 39

https://doi.org/10.1007/978-3-031-24755-2_3

Published: 03 February 2023 Publication History

Abstract

Recently, pre-trained language models (PTM) have achieved great success on ad hoc search. However, the performance decline in low-resource scenarios demonstrates the capability of PTM has not been inspired fully. As a novel paradigm to apply PTM to downstream tasks, prompt learning is a feasible scheme to boost PTM’s performance by aligning the pre-training task and downstream task. This paper investigates the effectiveness of the standard prompt learning paradigm on the ad hoc search task. Based on various PTMs, two types of prompts are tailored for the ad hoc search task. Overall experimental results on the MS Marco dataset show the credible better performance of our prompt learning method than fine-tuning based methods and another previous prompt learning based model. Experiments conducted in various resource scenarios show the stability of prompt learning. RoBERTa and T5 deliver better results compared to BM25 using 100 training queries utilizing prompt learning, while fine-tuning based methods need more data. Further analysis shows the significance of the uniformity of tasks’ format and adding continuous tokens into training in our prompt learning method.

References

[1]

Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

[2]

Nogueira, R., Cho, K.: Passage re-ranking with BERT. arXiv preprint arXiv:1901.04085 (2019)

[3]

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021)

[4]

Zhang, X., Yates, A., Lin, J.: A little bit is worse than none: ranking with limited training data. In: Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing, pp. 107–112 (2020)

[5]

Petroni, F., et al.: Language models as knowledge bases? arXiv preprint arXiv:1909.01066 (2019)

[6]

Brown T et al. Language models are few-shot learners Adv. Neural. Inf. Process. Syst. 2020 33 1877-1901

[7]

Liu, X., et al.: GPT understands, too. arXiv preprint arXiv:2103.10385 (2021)

[8]

Han, X., Zhao, W., Ding, N., Liu, Z., Sun, M.: PTR: prompt tuning with rules for text classification. arXiv preprint arXiv:2105.11259 (2021)

[9]

Nogueira, R., Jiang, Z., Lin, J.: Document ranking with a pretrained sequence-to-sequence model. arXiv preprint arXiv:2003.06713 (2020)

[10]

Raffel C et al. Exploring the limits of transfer learning with a unified text-to-text transformer J. Mach. Learn. Res. 2020 21 140 1-67

[11]

Hu, X., Yu, S., Xiong, C., Liu, Z., Liu, Z., Yu, G.: P ranker: mitigating the gaps between pre-training and ranking fine-tuning with prompt-based learning and pre-finetuning. arXiv preprint arXiv:2205.01886 (2022)

[12]

Williams, A., Nangia, N., Bowman, S.R.: A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 (2017)

[13]

Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

[14]

Nguyen, T., et al.: MS MARCO: a human generated machine reading comprehension dataset. In: CoCo@ NIPS (2016)

[15]

Robertson SE, Walker S, Jones S, Hancock-Beaulieu MM, Gatford M, et al. Okapi at TREC-3 NIST Spec. Publ. 1995 109 109

[16]

Karpukhin, V., et al.: Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 (2020)

[17]

Xiong, L., et al.: Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.00808 (2020)

[18]

Qu, Y., et al.: RocketQA: an optimized training approach to dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2010.08191 (2020)

[19]

Li, C., Yates, A., MacAvaney, S., He, B., Sun, Y.: PARADE: passage representation aggregation for document reranking. arXiv preprint arXiv:2008.09093 (2020)

[20]

Dai, Z., Callan, J.: Deeper text understanding for IR with contextual neural language modeling. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 985–988 (2019)

[21]

Nogueira, R., Yang, W., Cho, K., Lin, J.: Multi-stage document ranking with BERT. arXiv preprint arXiv:1910.14424 (2019)

[22]

Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E., Singh, S.: Autoprompt: eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 (2020)

[23]

Li, X.L., Liang, P.: Prefix-tuning: optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)

[24]

Hambardzumyan, K., Khachatrian, H., May, J.: WARP: word-level adversarial reprogramming. arXiv preprint arXiv:2101.00121 (2021)

[25]

Ding, N., et al.: OpenPrompt: an open-source framework for prompt-learning. arXiv preprint arXiv:2111.01998 (2021)

[26]

Wolf, T., et al.: HuggingFace’s transformers: state-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)

Cited By

Li HAi QChen JDong QWu YLiu YChen CTian QChen HDuh WHuang HKato MMothe JPoblete B(2023)SAILER: Structure-aware Pre-trained Language Model for Legal Case RetrievalProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591761(1035-1044)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591761
Li HAi QZhan JMao JLiu YLiu ZCao ZChen HDuh WHuang HKato MMothe JPoblete B(2023)Constructing Tree-based Index for Efficient and Effective Dense RetrievalProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591651(131-140)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591651

Index Terms

Enhance Performance of Ad-hoc Search via Prompt Learning
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
    2. Retrieval models and ranking

Index terms have been assigned to the content through auto-classification.

Recommendations

Axiomatically Regularized Pre-training for Ad hoc Search
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recently, pre-training methods tailored for IR tasks have achieved great success. However, as the mechanisms behind the performance improvement remain under-investigated, the interpretability and robustness of these pre-trained models still need to be ...
A fine-grained self-adapting prompt learning approach for few-shot learning with pre-trained language models
Abstract
Pre-trained language models have demonstrated remarkable performance in few-shot learning through the emergence of “prompt-based learning” methods, where the performance of these tasks highly rely on the quality of prompts. Existing prompt ...
Highlights
- We present the weak consistency assumption that the prompts in a task are consistent in the whole task but diverse in individuals, which allows us to learn example-specific templates.
- We propose to train a prompt generator with the ...
Personalized Prompt Learning for Explainable Recommendation
Providing user-understandable explanations to justify recommendations could help users better understand the recommended items, increase the system’s ease of use, and gain users’ trust. A typical approach to realize it is natural language generation. ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

Information Retrieval: 28th China Conference, CCIR 2022, Chongqing, China, September 16–18, 2022, Revised Selected Papers

Sep 2022

116 pages

ISBN:978-3-031-24754-5

DOI:10.1007/978-3-031-24755-2

Editors:
Yi Chang
Dingxin Building C403, Jilin University, Changchun, China
,
Xiaofei Zhu
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China

© Springer Nature Switzerland AG 2023.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 03 February 2023

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li HAi QChen JDong QWu YLiu YChen CTian QChen HDuh WHuang HKato MMothe JPoblete B(2023)SAILER: Structure-aware Pre-trained Language Model for Legal Case RetrievalProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591761(1035-1044)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591761
Li HAi QZhan JMao JLiu YLiu ZCao ZChen HDuh WHuang HKato MMothe JPoblete B(2023)Constructing Tree-based Index for Efficient and Effective Dense RetrievalProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591651(131-140)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591651

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents