research-article

Learning to Answer Complex Questions over Knowledge Bases with Query Composition

Authors:

Nikita Bhutani,

H V JagadishAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 739 - 748

https://doi.org/10.1145/3357384.3358033

Published: 03 November 2019 Publication History

Abstract

Recent years have seen a surge of knowledge-based question answering (KB-QA) systems which provide crisp answers to user-issued questions by translating them to precise structured queries over a knowledge base (KB). A major challenge in KB-QA is bridging the gap between natural language expressions and the complex schema of the KB. As a result, existing methods focus on simple questions answerable with one main relation path in the KB and struggle with complex questions that require joining multiple relations. We propose a KB-QA system, TextRay, which answers complex questions using a novel decompose-execute-join approach. It constructs complex query patterns using a set of simple queries. It uses a semantic matching model which is able to learn simple queries using implicit supervision from question-answer pairs, thus eliminating the need for complex query patterns. Our proposed system significantly outperforms existing KB-QA systems on complex questions while achieving comparable results on simple questions.

References

[1]

Abdalghani Abujabal, Mohamed Yahya, Mirek Riedewald, and Gerhard Weikum. 2017. Automated template generation for question answering over knowledge graphs. In Proc. WWW '17. International World Wide Web Conferences Steering Committee, 1191--1200.

Digital Library

[2]

Junwei Bao, Nan Duan, Zhao Yan, Ming Zhou, and Tiejun Zhao. 2016. Constraint-based question answering with knowledge graph. In Proc. COLING '16 . 2503--2514.

[3]

Hannah Bast and Elmar Haussmann. 2015. More accurate question answering on freebase. In Proc. CIKM '15. ACM, 1431--1440.

Digital Library

[4]

Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic Parsing on Freebase from Question-Answer Pairs. In Proc. EMNLP '13 . 1533--1544.

[5]

Jonathan Berant and Percy Liang. 2014. Semantic Parsing via Paraphrasing. In Proc. ACL' 14. ACL, 1415--1425.

[6]

Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proc. SIGMOD '08, Jason Tsong-Li Wang (Ed.). ACM, 1247--1250.

Digital Library

[7]

Antoine Bordes, Jason Weston, and Nicolas Usunier. 2014. Open Question Answering with Weakly Supervised Embedding Models. In Proc. ECML '14, Toon Calders, Floriana Esposito, Eyke Hü llermeier, and Rosa Meo (Eds.), Vol. 8724. Springer, 165--180.

Digital Library

[8]

Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung-won Hwang, and Wei Wang. 2017. KBQA: learning question answering over QA corpora and knowledge bases. Proc. VLDB '17, Vol. 10, 5 (2017), 565--576.

Digital Library

[9]

Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, and Andrew McCallum. 2017. Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv preprint arXiv:1711.05851 (2017).

[10]

Yansong Feng, Songfang Huang, Dongyan Zhao, et almbox. 2016. Hybrid question answering over knowledge base and free text. In Proc. COLING '16 . 2397--2407.

[11]

Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching Machines to Read and Comprehend. In NIPS .

[12]

Johannes Hoffart, Fabian M Suchanek, Klaus Berberich, and Gerhard Weikum. 2013. YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia. Artif. Intell., Vol. 194 (2013), 28--61.

Digital Library

[13]

Sen Hu, Lei Zou, Jeffrey Xu Yu, Haixun Wang, and Dongyan Zhao. 2018. Answering Natural Language Questions by Subgraph Matching over Knowledge Graphs. IEEE Trans. Knowl. Data Eng., Vol. 30, 5 (2018), 824--837.

[14]

Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang. 2017. Search-based Neural Structured Learning for Sequential Question Answering. In Proc. ACL '17 . 1821--1831.

[15]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings .

[16]

Fei Li and HV Jagadish. 2014. Constructing an interactive natural language interface for relational databases. Proc. VLDB '14, Vol. 8, 1 (2014), 73--84.

Digital Library

[17]

Chen Liang, Jonathan Berant, Quoc Le, Kenneth D Forbus, and Ni Lao. 2016. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. arXiv preprint arXiv:1611.00020 (2016).

[18]

Kangqi Luo, Fengli Lin, Xusheng Luo, and Kenny Q. Zhu. 2018. Knowledge Base Question Answering via Encoding of Complex Query Graphs. In Proc. EMNLP '18 . 2185--2194.

[19]

Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. In Proc. EMNLP '15 . 1412--1421.

[20]

Haoruo Peng, Ming-Wei Chang, and Wen-tau Yih. 2017a. Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision. In Proc. EMNLP '17. 2368--2378.

[21]

Haoruo Peng, Ming-Wei Chang, and Wen-tau Yih. 2017b. Maximum Margin Reward Networks for Learning from Explicit and Implicit Supervision. In Proc. EMNLP '17. 2368--2378.

[22]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proc. EMNLP '14. 1532--1543.

[23]

Linfeng Song, Zhiguo Wang, Mo Yu, Yue Zhang, Radu Florian, and Daniel Gildea. 2018. Exploring graph-structured passage representation for multi-hop reading comprehension with graph neural networks. arXiv preprint arXiv:1809.02040 (2018).

[24]

Alon Talmor and Jonathan Berant. 2018. The Web as a Knowledge-Base for Answering Complex Questions. In Proc. NAACL '18 . 641--651.

[25]

Christina Unger, Lorenz Bühmann, Jens Lehmann, Axel-Cyrille Ngonga Ngomo, Daniel Gerber, and Philipp Cimiano. 2012. Template-based question answering over RDF data. In Proc. WWW '12. ACM, 639--648.

Digital Library

[26]

Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In Proc. EMNLP '17 . 564--573.

[27]

Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang, and Dongyan Zhao. 2016. Question Answering on Freebase via Relation Extraction and Textual Evidence. In Proc. ACL '16 .

[28]

Yi Yang and Ming-Wei Chang. 2015. S-MART: Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking. In Proc. ACL'15. 504--513.

[29]

Xuchen Yao and Benjamin Van Durme. 2014. Information Extraction over Structured Data: Question Answering with Freebase. In Proc. ACL '14. ACL, 956--966.

[30]

Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base. In Proc. ACL '15. 1321--1331.

[31]

Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. In Proc. ACL '16, Vol. 2. 201--206.

[32]

Pengcheng Yin, Nan Duan, Ben Kao, Junwei Bao, and Ming Zhou. 2015. Answering questions with complex semantic constraints on open knowledge bases. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 1301--1310.

Digital Library

[33]

Mo Yu, Wenpeng Yin, Kazi Saidul Hasan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2017. Improved neural relation detection for knowledge base question answering. arXiv preprint arXiv:1704.06194 (2017).

[34]

Weiguo Zheng, Jeffrey Xu Yu, Lei Zou, and Hong Cheng. 2018. Question answering over knowledge graphs: question understanding via template decomposition. Proc. VLDB '18, Vol. 11, 11 (2018), 1373--1386.

Digital Library

[35]

Lei Zou, Ruizhe Huang, Haixun Wang, Jeffrey Xu Yu, Wenqiang He, and Dongyan Zhao. 2014. Natural language question answering over RDF: a graph data driven approach. In Proc. SIGMOD '14 . ACM, 313--324.

Digital Library

Cited By

Yu RWang DJia XJiang ZZhang Z(2025)A Review on the Large Language Model Augmented Knowledge Graph Question Answer: Task, Model, Advance and OutlookProceedings of the 2023 International Conference on Wireless Communications, Networking and Applications10.1007/978-981-96-2409-6_33(333-347)Online publication date: 27-Feb-2025
https://doi.org/10.1007/978-981-96-2409-6_33
Lan MXia YZhou GHuang NLi ZWu H(2024)LLM4QA: Leveraging Large Language Model for Efficient Knowledge Graph Reasoning with SPARQL QueryJournal of Advances in Information Technology10.12720/jait.15.10.1157-116215:10(1157-1162)Online publication date: 2024
https://doi.org/10.12720/jait.15.10.1157-1162
Zhang ZWen LZhao WSerra ESpezzano F(2024)A GAIL Fine-Tuned LLM Enhanced Framework for Low-Resource Knowledge Graph Question AnsweringProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679753(3300-3309)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679753
Show More Cited By

Index Terms

Learning to Answer Complex Questions over Knowledge Bases with Query Composition
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
      1. Query representation
    2. Retrieval tasks and goals
      1. Question answering

Recommendations

Leveraging Dual Encoder Models for Complex Question Answering over Knowledge Bases
PRICAI 2023: Trends in Artificial Intelligence
Abstract
Knowledge-based question answering is a hot topic in Natural Language Processing (NLP), especially in addressing complex questions. Existing methods, which transform complex questions into query graphs, often struggle with low-quality graphs. To ...
Question Answering over Knowledge Graphs with Query Path Generation
Knowledge Science, Engineering and Management
Abstract
Knowledge graphs have been applied in question answering. Many researchers have proposed methods based on query graph generation, but there are some defects such as high cost of query graph generation and large search scope of knowledge graphs. ...
Answering Complex Questions on Knowledge Graphs
Knowledge Science, Engineering and Management
Abstract
The topic of knowledge-based question answering ( $KBQA$ ) has attracted wide attention for a long period. A series of techniques have been developed, especially for simple questions. To answer complex questions, most existing approaches apply a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
1,044
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)1

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu RWang DJia XJiang ZZhang Z(2025)A Review on the Large Language Model Augmented Knowledge Graph Question Answer: Task, Model, Advance and OutlookProceedings of the 2023 International Conference on Wireless Communications, Networking and Applications10.1007/978-981-96-2409-6_33(333-347)Online publication date: 27-Feb-2025
https://doi.org/10.1007/978-981-96-2409-6_33
Lan MXia YZhou GHuang NLi ZWu H(2024)LLM4QA: Leveraging Large Language Model for Efficient Knowledge Graph Reasoning with SPARQL QueryJournal of Advances in Information Technology10.12720/jait.15.10.1157-116215:10(1157-1162)Online publication date: 2024
https://doi.org/10.12720/jait.15.10.1157-1162
Zhang ZWen LZhao WSerra ESpezzano F(2024)A GAIL Fine-Tuned LLM Enhanced Framework for Low-Resource Knowledge Graph Question AnsweringProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679753(3300-3309)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679753
Xinyu HJian YGang X(2024)Knowledge-injected Stepwise Reasoning on Complex KBQA2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650658(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650658
Ren YLi XLiu XZhang R(2024)Few-Shot KBQA Method Based on Multi-Task Learning2024 IEEE International Conference on Big Data and Smart Computing (BigComp)10.1109/BigComp60711.2024.00043(226-233)Online publication date: 18-Feb-2024
https://doi.org/10.1109/BigComp60711.2024.00043
Pramanik SAlabi JRoy RWeikum G(2024)Uniqorn: Unified question answering over RDF knowledge graphs and natural language textJournal of Web Semantics10.1016/j.websem.2024.10083383(100833)Online publication date: Dec-2024
https://doi.org/10.1016/j.websem.2024.100833
Srivastava NPerevalov AKuchelev DMoussallem DNgonga Ngomo ABoth A(2023)Lingua Franca – Entity-Aware Machine Translation Approach for Question Answering over Knowledge GraphsProceedings of the 12th Knowledge Capture Conference 202310.1145/3587259.3627567(122-130)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3587259.3627567
Zhang DLiu ZJia WWu FLiu HTan J(2023)Dual Attention Graph Convolutional Network for Relation ExtractionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.3289879(1-14)Online publication date: 2023
https://doi.org/10.1109/TKDE.2023.3289879
Lan YHe GJiang JJiang JZhao WWen J(2023)Complex Knowledge Base Question Answering: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.322385835:11(11196-11215)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TKDE.2022.3223858
Liu PQian BSun QZhao L(2023)Prompt-WNQA: A prompt-based complex question answering for wireless network over knowledge graphComputer Networks10.1016/j.comnet.2023.110014236(110014)Online publication date: Nov-2023
https://doi.org/10.1016/j.comnet.2023.110014
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten