Prompt-based Few-shot Learning for Table-based Fact Verification
Authors: 
Lei Hou, Yubo Liu, Jie Wu, Mengshu HouAuthors Info & Claims
Lei Hou, Yubo Liu, Jie Wu, Mengshu HouAuthors Info & ClaimsPages 14 - 19
Abstract
Natural language processing has been a hot topic of research, but existing research is mainly limited to unstructured information such as natural language sentences and documents, and less research has been done on structured information such as tables. The main object of this paper is the table-based fact verification task, under which there is only one TABFACT dataset. Most of the existing methods on this dataset are based on pre-trained models and need to be fine-tuned again if a new dataset appears. And some previous work on natural language sentences has shown that prompt approach can achieve good performance with few samples. Therefore, in this paper, we adopt the prompt approach for experiments on the table fact detection task by manually designing templates for hinting the pre-trained model. Meanwhile, to enhance the generalization of the model, we introduce a multi-pair mapping relationship in the Answer Engineering phase. Experiments on the TABFACT dataset show that using the prompt method for table-based fact verification task in the case of few samples can be effective, providing a new way for optimizing table-related tasks in the case of few samples.
References
[1]
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv:1909.02164(2019).
[2]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).
[3]
Tianyu Gao, Adam Fisch, and Danqi Chen. 2020. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723(2020).
[4]
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2021. Ppt: Pre-trained prompt tuning for few-shot learning. arXiv preprint arXiv:2109.04332(2021).
[5]
Jonathan Herzig, Paweł Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Martin Eisenschlos. 2020. TaPas: Weakly supervised table parsing via pre-training. arXiv preprint arXiv:2004.02349(2020).
[6]
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586(2021).
[7]
Robert L Logan IV, Ivana Balažević, Eric Wallace, Fabio Petroni, Sameer Singh, and Sebastian Riedel. 2021. Cutting down on prompts and parameters: Simple few-shot learning with language models. arXiv preprint arXiv:2106.13353(2021).
[8]
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786(2021).
[9]
Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305(2015).
[10]
Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2019. Deep contextualized word representations. (2019).
[11]
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, 2018. Improving language understanding by generative pre-training. (2018).
[12]
Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, and Zhiyuan Liu. 2019. NumNet: Machine reading comprehension with numerical reasoning. arXiv preprint arXiv:1910.06701(2019).
[13]
Timo Schick and Hinrich Schütze. 2020a. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676(2020).
[14]
Timo Schick and Hinrich Schütze. 2020b. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.07118(2020).
[15]
Qi Shi, Yu Zhang, Qingyu Yin, and Ting Liu. 2020. Learn to combine linguistic and symbolic information for table-based fact verification. In Proceedings of the 28th International Conference on Computational Linguistics. 5335–5346.
[16]
Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021. Improving and simplifying pattern exploiting training. arXiv preprint arXiv:2103.11955(2021).
[17]
Xiaoyu Yang, Feng Nie, Yufei Feng, Quan Liu, Zhigang Chen, and Xiaodan Zhu. 2020. Program enhanced fact verification with verbalization and graph attention network. arXiv preprint arXiv:2010.03084(2020).
[18]
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems 32 (2019).
[19]
Hongzhi Zhang, Yingyao Wang, Sirui Wang, Xuezhi Cao, Fuzheng Zhang, and Zhongyuan Wang. 2020. Table fact verification with structure-aware transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1624–1629.
[20]
Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. 2021. Differentiable prompt makes pre-trained language models better few-shot learners. arXiv preprint arXiv:2108.13161(2021).
[21]
Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103(2017).
Index Terms
- Prompt-based Few-shot Learning for Table-based Fact Verification
Recommendations
Joint contrastive learning for prompt-based few-shot language learners
AbstractThe combination of prompt learning and contrastive learning has recently been a promising approach to few-shot learning in NLP field. However, most of these studies only focus on the semantic-level relevance and intra-class information of data in ...
Domain-control prompt-driven zero-shot relational triplet extraction
AbstractZero-shot relational triplet extraction is a vital solution to the problem of fact extracted from unstructured text without labeled training data. In the task, the data is divided into seen and unseen relations for training and prediction, ...
Highlights- Prompts are able to control the output domain, guiding the extraction of unseen triplets.
- Potential relations can be determined according to semantic matching between sentences and relation description text.
- A two-stage framework ...
OpenTFV: An Open Domain Table-Based Fact Verification System
SIGMOD '22: Proceedings of the 2022 International Conference on Management of DataThe prevalence of misinformation, both online and offline, has prompted a great demand of fact verification. Table-based fact verification aims to check whether a textual claim is supported or refuted based on relational tables. However, most of the ...
Comments
Information & Contributors
Information
Published In
December 2022
406 pages
ISBN:9781450399067
DOI:10.1145/3578741
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 March 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
MLNLP 2022
MLNLP 2022: 2022 5th International Conference on Machine Learning and Natural Language Processing
December 23 - 25, 2022
Sanya, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 120Total Downloads
- Downloads (Last 12 months)60
- Downloads (Last 6 weeks)1
Reflects downloads up to 01 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format