Two minutes NLP — Quick Intro to Knowledge Base Question Answering
KBQA, Semantic Parsing-based Methods, and Information Retrieval-based Methods.
Hello fellow NLP enthusiasts! As we previously saw how question answering over documents is done, we now jump into a different kind of question answering leveraging knowledge bases. Enjoy! 😄
What is Knowledge Base Question Answering
Knowledge base question answering (KBQA) aims to answer a natural language question over a knowledge base (KB) as its knowledge source.
A knowledge base (KB) is a structured database that contains a collection of facts in the form <subject, relation, object> , where each fact can have properties attached called qualifiers.
For example, the sentence “Barack Obama got married to Michelle Obama on 3 October 1992 at Trinity United Church” can be represented by the tuple <Barack Obama, Spouse, Michelle Obama> , with the qualifiers start time = 3 October 1992 and place of marriage = Trinity United Church .
Popular knowledge bases are DBpedia and WikiData.
Simple questions vs complex questions
Early works on KBQA focused on simple question answering, where there’s only a single fact involved. For example, “Where was JK Rowling born?” is a simple question that can be answered using just the fact <J.K. Rowling, birthplace, United Kingdom>.
Recently, attention shifted to answering complex questions. Generally, complex questions involve multi-hop reasoning over the KB, constrained relations, numerical operations, or some combination of the above.
Let’s see an example of complex KBQA with the question “Who is the first wife of the TV producer that was nominated for The Jeff Probst Show?”. This question requires:
- Constrained relations: We are looking for the TV producer that was nominated for The Jeff Probst Show, thus we are looking for an entity with a
nomineelink to theThe Jeff Probst Showand that is a TV producer. - Multi-hop reasoning: Once we find the TV producer, we need to find his wives.
- Numerical operations: Once we find the TV producer's wives, we are looking for the first wife, thus we need to compare numbers and generate a ranking.
KBQA approaches
There are two mainstream approaches for complex KBQA. Both these two approaches start by recognizing the subject in the question and linking it to an entity in the KB, which will be called topic entity. Then, they derive the answers within the KB neighborhood of the topic entity:
- By executing a parsed logic form, typical of semantic parsing-based methods (SP-based methods). It follows a parse-then-execute paradigm.
- By reasoning in a question-specific graph extracted from the KB and ranking all the entities in the extracted graph based on their relevance to the question, typical of information retrieval-based methods (IR-based methods). It follows a retrieval-and-rank paradigm.
Semantic Parsing-based Methods
This category of methods aims at parsing a natural language utterance into logic forms. They predict answers via the following steps:
- Parse the natural language question into an uninstantiated logic form (e.g. a SPARQL query template), which is a syntactic representation of the question without the grounding of entities and relations.
- The logic form is then instantiated and validated by conducting some semantic alignments to structured KBs via KB grounding (obtaining, for example, an executable SPARQL query).
- The parsed logic form is executed against KBs to generate predicted answers.
Information Retrieval-based Methods
IR-based methods directly retrieve and rank answers from the KBs considering the information conveyed in the questions. They consist of the following steps:
- Starting from the topic entity, the system first extracts a question-specific graph from KBs, ideally including all question-related entities and relations as nodes and edges.
- Next, the system encodes input questions into vectors representing reasoning instructions.
- A graph-based reasoning module conducts semantic matching via vector-based computation to propagate and then aggregate the information along the neighboring entities within the graph.
- An answer ranking module is utilized to rank the entities in the graph according to the reasoning status at the end of the reasoning phase. The top-ranked entities are predicted as the answers to the question.
Pros and Cons of the two approaches
Semantic parsing-based methods produce a more interpretable reasoning process thanks to logic forms. However, their heavy reliance on the design of logic forms and parsing modules is usually a bottleneck for performance improvements.
IR-based methods naturally fit into popular end-to-end training, making them easy to train. However, the black-box style of the reasoning model makes the intermediate reasoning less interpretable.
Conclusions and next steps
In this article, we learned what is knowledge-based question answering (KBQA), and what are the two main approaches for managing complex questions.
Possible next steps are:
- Learn more about popular knowledge bases like DBpedia and WikiData.
- Learn about the SPARQL query language, useful to query knowledge bases.
