Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

A novel technique using graph neural networks and relevance scoring to improve the performance of knowledge graph-based question answering systems

  • Research
  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

A Knowledge Graph-based Question Answering (KGQA) system attempts to answer a given natural language question using a knowledge graph (KG) rather than from text data. The current KGQA methods attempt to determine whether there is an explicit relationship between the entities in the question and a well-structured relationship between them in the KG. However, such strategies are difficult to build and train, limiting their consistency and versatility. The use of language models such as BERT has aided in the advancement of natural language question answering. In this paper, we present a novel Graph Neural Network(GNN) based approach with relevance scoring for improving KGQA. GNNs use the weight of nodes and edges to influence the information propagation while updating the node features in the network. The suggested method comprises subgraph construction, weighing of nodes and edges, and pruning processes to obtain meaningful answers. BERT-based GNN is used to build subgraph node embeddings. We tested the influence of weighting for both nodes and edges and observed that the system performs better for weighted graphs than unweighted graphs. Additionally, we experimented with several GNN convolutional layers and obtainined improved results by combining GENeralised Graph Convolution (GENConv) with node weights for simple questions. Extensive testing on benchmark datasets confirmed the effectiveness of the proposed model in comparison to state-of-the-art KGQA systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 2
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of supporting data

The data used in the current study are available from the websites, https://github.com/yuyuz/MetaQA and http://aka.ms/WebQSP.

Notes

  1. https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens

  2. https://github.com/yuyuz/MetaQA

  3. http://aka.ms/WebQSP

  4. https://pytorch-geometric.readthedocs.io/en/latest/

  5. https://networkx.org/

  6. https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html

References

  • Bai, Y., Lv, X., et al. (2023). Answering complex logical queries on knowledge graphs via query computation tree optimization. JMLR.org, ICML’23, https://dl.acm.org/doi/10.5555/3618408.3618470

  • Bordes, A., Usunier, N., et al. (2015). Large–scale simple question answering with memory networks. CoRR abs/1506.02075. arXiv:1506.02075

  • Daniel, J., & James, M. (2019). Speech and Language Processing: An Introduction to Natural Language Processing. Computational Linguistics and Speech Recognition. 3rd Edition Draft. Prentice Hall, New Jersey., Chapter 25

  • Devlin J, Chang MW, et al. (2019) Bert: Pre–training of deep bidirectional transformers for language understanding. In: North American Chapter of the ACL, https://api.semanticscholar.org/CorpusID:52967399

  • Feng, Y., Chen, X., et al. (2020). Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proceedings of the 2020 Conference EMNLP. ACL, Online, pp 1295–1309, https://doi.org/10.18653/v1/2020.emnlp-main.99

  • Hamilton, W. L. (2020). Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3), 1–159.

    Article  Google Scholar 

  • Hao, C., Xie, M., et al. (2022). ACENet: Attention guided commonsense reasoning on hybrid knowledge graph. In: Proceedings of the Conference on EMNLP. ACL, Abu Dhabi, UAE, pp. 8461–8471, https://doi.org/10.18653/v1/2022.emnlp-main.579

  • He, G., Lan, Y., et al. (2021). Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, WSDM ’21, p 553–561, https://doi.org/10.1145/3437963.3441753

  • Hogan, A., Blomqvist, et al. (2021). Knowledge graphs. ACM Comput Surv 54(4). https://doi.org/10.1145/3447772

  • Huang, X., Zhang, J., et al. (2019). Knowledge graph embedding based question answering. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM ’19, pp. 105–113, https://doi.org/10.1145/3289600.3290956

  • Huertas, G., Alvaro, et al. (2023). Exploring dimensionality reduction techniques in multilingual transformers. Cognitive Computation, 15(2), 590–612. https://doi.org/10.1007/s12559-022-10066-8

  • Ji, S., Pan, S., et al. (2022). A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 33(2), 494–514. https://doi.org/10.1109/TNNLS.2021.3070843

    Article  MathSciNet  Google Scholar 

  • Jin, W., Zhao, B., et al. (2023). Improving embedded knowledge graph multi-hop question answering by introducing relational chain reasoning. Data Mining and Knowledge Discovery, 37(1), 255–288. https://doi.org/10.1007/s10618-022-00891-8

    Article  Google Scholar 

  • Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Y. Bengio & Y. LeCun (Eds.), 3rd International Conference on Learning Representations. ICLR.

    Google Scholar 

  • Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations, https://openreview.net/forum?id=SJU4ayYgl

  • Kwok, C. C. T., Etzioni, O., & Weld, D. S. (2001). Scaling question answering to the web. In: Proceedings of the 10th International Conference on World Wide Web. Association for Computing Machinery, New York, NY, USA, WWW ’01, p 150–161, https://doi.org/10.1145/371920.371973

  • Li G, Xiong C, Thabet AK, et al (2020) Deepergcn: All you need to train deeper gcns. arXiv:2006.07739. https://api.semanticscholar.org/CorpusID:219687460

  • Ma, R., Han, X., et al. (2023). Modeling and querying temporal rdf knowledge graphs with relational databases. J Intell Inf Syst, 61, 569–609. https://doi.org/10.1007/s10844-023-00780-6

    Article  Google Scholar 

  • Maćkiewicz, A., & Ratajczak, W. (1993). Principal components analysis (pca). Computers & Geosciences, 19(3), 303–342. https://doi.org/10.1016/0098-3004(93)90090-R

    Article  Google Scholar 

  • Miller, A., Fisch, A., et al. (2016). Key–value memory networks for directly reading documents. In: Proceedings of the 2016 Conference on EMNLP. ACL, pp. 1400–1409. https://doi.org/10.18653/v1/D16-1147

  • Mital, P., Agarwal, S., Neti, B., et al. (2018). Graph based question answering system. CoRR abs/1812.01828. arXiv:1812.01828

  • Peng, Z., Yu, H., et al. (2022). Path-based reasoning with k-nearest neighbor and position embedding for knowledge graph completion. J Intell Inf Syst, 58, 513–533. https://doi.org/10.1007/s10844-021-00671-8

    Article  Google Scholar 

  • Qiu, Y., Wang, Y., Jin, X., et al. (2020) Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision. In: Proceedings of the 13th International Conference on Web Search and Data Mining. Association for Computing Machinery, pp. 474–482, https://doi.org/10.1145/3336191.3371812

  • Sacenti, J. A. P., & Rea, Fileto. (2022). Knowledge graph summarization impacts on movie recommendations. J Intell Inf Syst, 58, 43–66. https://doi.org/10.1007/s10844-021-00650-z

    Article  Google Scholar 

  • Saxena, A., Tripathi, A., & Talukdar, P. (2020). Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp 4498–4507, https://doi.org/10.18653/v1/2020.acl-main.412

  • Scarselli, F., Gori, M., et al. (2009). The graph neural network model. IEEE Transactions on Neural Networks, 20(1), 61–80. https://doi.org/10.1109/TNN.2008.2005605

    Article  Google Scholar 

  • Schlichtkrull, M., Kipf, T. N., Bloem, P., et al. (2018). Modeling relational data with graph convolutional networks. In: The Semantic Web. Springer International Publishing, Cham, pp. 593–607, https://doi.org/10.1007/978-3-319-93417-4_38

  • Sen, P., Oliya, A., Saffari, A. (2021). Expanding end-to-end question answering on differentiable knowledge graphs with intersection. In: Proceedings of the Conference on EMNLP. ACL, pp. 8805–8812, https://doi.org/10.18653/v1/2021.emnlp-main.694

  • Shi, J., Cao, S., et al. (2021). TransferNet: An effective and transparent framework for multi-hop question answering over relation graph. In: Proceedings of the Conference on EMNLP, pp. 4149–4158, https://doi.org/10.18653/v1/2021.emnlp-main.341

  • Shi, Y., Huang, Z., et al. (2020). Masked label prediction: Unified massage passing model for semi-supervised classification. CoRR abs/2009.03509. arXiv:2009.03509

  • Sun, H., Dhingra, B., et al. (2018). Open domain question answering using early fusion of knowledge bases and text. In: Proceedings of the 2018 Conference on EMNLP. ACL, pp. 4231–4242, https://doi.org/10.18653/v1/D18-1455

  • Sun, H., Bedrax, T., et al. (2019). PullNet: Open domain question answering with iterative retrieval on knowledge bases and text. In: Proceedings of the Conference on EMNLP and the 9th IJCNLP, pp. 2380–2390, https://doi.org/10.18653/v1/D19-1242

  • Sun, Y., Shi, Q., Qi, L., et al. (2022). JointLK: Joint reasoning with language models and knowledge graphs for commonsense question answering. In: Proceedings of the Conference of the North American Chapter of the ACL: Human Language Technologies, pp. 5049–5060, https://doi.org/10.18653/v1/2022.naacl-main.372

  • Thambi, S. V., & ReghuRaj, P. C. (2022a). Graph based document model and its application in keyphrase extraction. In: IEEE SPICES, pp 92–98, https://doi.org/10.1109/SPICES52834.2022.9774141

  • Thambi, S. V., & ReghuRaj, P. C. (2022b). Towards improving the performance of question answering system using knowledge graph - a survey. In: Second ICAIS), pp 672–679, https://doi.org/10.1109/ICAIS53314.2022.9742802

  • Unger, C., Bühmann, L., Lehmann, J., et al. (2012). Template-based question answering over rdf data. In: Proceedings of the 21st International Conference on World Wide Web. Association for Computing Machinery, New York, NY, USA, WWW ’12, p 639–648, https://doi.org/10.1145/2187836.2187923

  • Vashishth, S., Sanyal, S., et al. (2020). Composition-based multi-relational graph convolutional networks. In: ICLR. OpenReview.net, http://dblp.uni-trier.de/db/conf/iclr/iclr2020.html#VashishthSNT20

  • Wang, X., Luo, M., Si, C., et al. (2022a). Answering complex questions on knowledge graphs. In: Knowledge Science, Engineering and Management. Springer International Publishing, Cham, pp. 187–200, https://doi.org/10.1007/978-3-031-10983-6_15

  • Wang, Y., Yasunaga, M., Ren, H., et al. (2022b). Vqa-gnn: Reasoning with multimodal semantic graph for visual question answering. arXiv:2205.11501. https://api.semanticscholar.org/CorpusID:263895473

  • Wu, G., He, Y., & Hu, X. (2018). Entity linking: An issue to extract corresponding entity with knowledge base. IEEE Access, 6, 6220–6231. https://doi.org/10.1109/ACCESS.2017.2787787

    Article  Google Scholar 

  • Xie, M., Hao, C., & Zhang, P. (2022). A sequential flow control framework for multi-hop knowledge base question answering. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 8450–8460, https://doi.org/10.18653/v1/2022.emnlp-main.578

  • Yasunaga, M., Ren, H., et al. (2021). QA-GNN: Reasoning with language models and knowledge graphs for question answering. In: Proceedings of the North American Chapter of the ACL: Human Language Technologies, pp 535–546, https://doi.org/10.18653/v1/2021.naacl-main.45

  • Yu, X., Liang, D., & Li, Q. (2023). Improved graphsvx for gnn explanations based on cross entropy. In: 2023 4th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), pp. 147–152, https://doi.org/10.1109/ICECAI58670.2023.10176786

  • Zhang, Y., Dai, H., Kozareva, Z., et al. (2018). Variational reasoning for question answering with knowledge graph. In: Proceedings of the 32 AAAI Conference on Artificial Intelligence, https://dl.acm.org/doi/pdf/10.5555/3504035.3504780

Download references

Acknowledgements

The authors would like to thank the CERD of APJ Abdul Kalam Technological University, Kerala.

Funding

This work was supported by Centre for Engineering Research and Development(CERD) of APJ Abdul Kalam Technological University, Kerala.

Author information

Authors and Affiliations

  1. Government Engineering College Kozhikode, Palakkad, India

    Sincy V. Thambi & P. C. Reghu Raj

  2. APJ Abdul Kalam Technological University, Thiruvananthapuram, Kerala, India

    Sincy V. Thambi & P. C. Reghu Raj

Authors
  1. Sincy V. Thambi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. C. Reghu Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Sincy V. Thambi, P.C. Reghu Raj served as scientific advisor.

Corresponding author

Correspondence to Sincy V. Thambi.

Ethics declarations

Ethics Approval

Not Applicable

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thambi, S.V., Reghu Raj, P.C. A novel technique using graph neural networks and relevance scoring to improve the performance of knowledge graph-based question answering systems. J Intell Inf Syst 62, 809–832 (2024). https://doi.org/10.1007/s10844-023-00839-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-023-00839-4

Keywords

Search

Navigation