Abstract
Knowledge base question answering (KBQA) is an important task that involves analyzing natural language questions and retrieving relevant answers from a knowledge base. To achieve this, Semantic Parsing (SP) is used to parse the question into a structured logical form, which is then executed to obtain the answer. Although different logical forms have unique advantages, existing methods only focus on a single logical form and do not consider the semantic consistency between different logical forms. In this paper, we address the issue of consistency in semantic parsing, which has not been explored before. We show that improving the semantic consistency between multiple logical forms can help increase the parsing performance. To address the consistency problem, we present a dynamic knowledge distillation framework for semantic parsing (DKD-SP). Our framework enables one logical form to learn some useful hidden knowledge from another, which improves the semantic consistency of different logical forms. Additionally, it dynamically adjusts the supervised weight of the hidden knowledge as the student model’s ability changes. We evaluate our approach on the KQA Pro dataset, and our experimental results confirm its effectiveness. Our method improves the overall accuracy of the seven types of questions by 0.57%, with notable improvements in the accuracy of Qualifier, Compare, and Count questions. Furthermore, in the compositional generalization scenario, the overall accuracy improved by 4.02%. Our codes are publicly available on https://github.com/zjtfo/SP_Consistency_By_KD.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Luo, Y., Yang, B., Xu, D., et al.: A survey: complex knowledge base question answering. In: Proceedings of ICICSE, pp. 46–52. IEEE (2022)
Cao, S., Shi, J., Pan, L., et al.: KQA Pro: a dataset with explicit compositional programs for complex question answering over knowledge base. In: Proceedings of ACL, pp. 6101–6119. ACL (2022)
Sun, Y., Zhang, L., Cheng, G., et al.: SPARQA: skeleton-based semantic parsing for complex questions over knowledge bases. In: Proceedings of AAAI, New York, USA, pp. 8952–8959. AAAI (2020)
Liang, P., Jordan, M.I., Klein, D.: Lambda dependency-based compositional semantics. In: Proceedings of CCL, Suzhou, China, pp. 389–446. ACL (2013)
Zhong, V., Xiong, C., Socher, R.: Seq2sql: generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103 (2017)
Veyseh, A., Dernoncourt, F., Dou, D., et al.: A joint model for definition extraction with syntactic connection and semantic consistency. In: Proceedings of AAAI, New York, USA, pp. 9098–9105. AAAI (2020)
Gangwoo, K., Hyunjae, K., Jungsoo, P., Jaewoo, K.: Learn to resolve conversational dependency: a consistency training framework for conversational question answering. In: Proceedings of IJCNLP, Bangkok, Thailand, pp. 6130–6141. ACL (2021)
Wang, A., Cho, K., Lewis, M.: Asking and answering questions to evaluate the factual consistency of summaries. In: Proceedings of ACL, Seattle, Washington, United States, pp. 5008–5020. ACL (2020)
Nan, F., et al.: Improving factual consistency of abstractive summarization via question answering. In: Proceedings of IJCNLP, Bangkok, Thailand, pp. 6881–6894. ACL (2021)
Qin, L., Xie, T., Huang, S., Chen, Q., Xu, X., Che, W.: Don’t be contradicted with anything! CI-ToD: towards benchmarking consistency for task-oriented dialogue system. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 2357–2367. ACL (2021)
Dharur, S., Tendulkar, P., Batra, D., Parikh, D., Selvaraju, R.: SOrT-ing VQA models: contrastive gradient learning for improved consistency. In: Proceedings of NAACL, Mexico City, pp. 3103–3111. ACL (2021)
Yang, S., Zhou, Q., Feng, D., et al.: Diversity and consistency: exploring visual question-answer pair generation. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 1053–1066. ACL (2021)
Lyu, X., Li, J., Gong, Z., et al.: Encouraging lexical translation consistency for document-level neural machine translation. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 3265–3277 (2021)
Gou, J., Yu, B., Maybank, S.J., et al.: Knowledge distillation: a survey. IJCV 129(6), 1789–1819 (2021)
Miller, A.H., Fisch, A., Dodge, J., Karimi, A.H., Bordes, A., Weston, J.: Key-value memory networks for directly reading documents. In: Proceedings of EMNLP, Austin, Texas, pp. 1400–1409. ACL (2016)
Saxena, A., Tripathi, A., Talukdar, P.: Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of ACL, Seattle, Washington, United States, pp. 4498–4507. ACL (2020)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Dey R., Salem F.M.: Gate-variants of gated recurrent unit (GRU) neural networks. In: Proceedings of MWSCAS, Boston, USA, pp. 1597-1600. IEEE (2017)
Dong, L., Lapata, M.: Language to logical form with neural attention. In: Proceedings of ACL, Berlin, Germany, pp. 33–43. ACL (2016)
Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of ACL, Seattle, Washington, United States, pp. 7871–7880. ACL (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of ICLR, San Diego, USA (2015)
Xu, K., Wu, L., Wang, Z., Yu, M., Chen, L., Sheinin, V.: Exploiting rich syntactic information for semantic parsing with graphtosequence model. In: Proceedings of EMNLP, Brussels, Belgium, pp. 918–924. ACL (2018)
Agrawal, P., Dalmia, A., Jain, P., Bansal, A., Mittal, A., Sankaranarayanan, K.: Unified semantic parsing with weak supervision. In: Proceedings of ACL, Florence, Italy, pp. 4801–4810. ACL (2019)
Cao, R., et al.: Unsupervised dual paraphrasing for two-stage semantic parsing. In: Proceedings of ACL, Seattle, Washington, United States, pp. 6806–6817. ACL (2020)
Aghaei, S., Raad, E., Fensel, A.: Question answering over knowledge graphs: a case study in tourism. IEEE Access 10, 69788–69801 (2022)
Oren, I., Herzig, J., Gupta, N., Gardner, M., Berant, J.: Improving compositional generalization in semantic parsing. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 2482–2495. ACL (2020)
Lukovnikov, D., Daubener, S., Fischer, A.: Detecting compositionally out-of-distribution examples in semantic parsing. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 591–598. ACL (2021)
Rashid, A., Lioutas, V., Rezagholizadeh, M.: Mate-kd: masked adversarial text, a companion to knowledge distillation. In: Proceedings of IJCNLP, Bangkok, Thailand, pp. 1062–1071. ACL (2021)
Wang, X., Jiang, Y., Bach, N., Wang, T., Huang, F., Tu, K.: Structure-level knowledge distillation for multilingual sequence labeling. In: Proceedings of ACL, Seattle, Washington, United States, pp. 3317–3330. ACL (2020)
Aguilar, G., Ling, Y., Zhang, Y., et al.: Knowledge distillation from internal representations. In: Proceedings of AAAI, New York, USA, pp. 7350–7357. AAAI (2020)
Mirzadeh, S.I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., Ghasemzadeh, H.: Improved knowledge distillation via teacher assistant. In: Proceedings of AAAI, New York, USA, pp. 5191–5198. AAAI (2020)
Yang, Z., Liu, J., Huang, J., et al.: Cross-modal contrastive distillation for instructional activity anticipation. In: Proceedings of ICPR, Montreal, QC, Canada, pp. 5002–5009. IEEE (2022)
Li, L., Lin, Y., Ren, S., Li, P., Zhou, J., Sun, X.: Dynamic knowledge distillation for pre-trained language models. In: Proceedings of EMNLP, Punta Cana, Dominican Republic, pp. 379–389. ACL (2021)
Ramamurthy, P., Aakur, S.N.: ISD-QA: iterative distillation of commonsense knowledge from general language models for unsupervised question answering. In: Proceedings of ICPR, Montreal, QC, Canada, pp. 1229–1235. IEEE (2022)
Luo, X., Chen, J., Song, T., et al.: Semi-supervised medical image segmentation through dual-task consistency. In: Proceedings of AAAI, pp. 8801–8809. AAAI (2021)
Panthaplackel, S., Li, J.J., Gligoric, M., et al.: Deep just-in-time inconsistency detection between comments and source code. In: Proceedings of AAAI, pp. 427–435. AAAI (2021)
Acknowledgement
This work is supported by the National Key R &D Program of China (2020AAA0105203).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zou, J., Cao, S., Wan, J., Hou, L., Xu, J. (2024). Improving the Consistency of Semantic Parsing in KBQA Through Knowledge Distillation. In: Song, X., Feng, R., Chen, Y., Li, J., Min, G. (eds) Web and Big Data. APWeb-WAIM 2023. Lecture Notes in Computer Science, vol 14333. Springer, Singapore. https://doi.org/10.1007/978-981-97-2387-4_25
Download citation
DOI: https://doi.org/10.1007/978-981-97-2387-4_25
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2386-7
Online ISBN: 978-981-97-2387-4
eBook Packages: Computer ScienceComputer Science (R0)