research-article

BayesKGR: Bayesian Few-Shot Learning for Knowledge Graph Reasoning

Authors:

Lifang HeAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing, Volume 22, Issue 6

Article No.: 160, Pages 1 - 21

https://doi.org/10.1145/3589183

Published: 17 June 2023 Publication History

Abstract

Reasoning over knowledge graphs (KGs) has received increasing attention recently due to its promising applications in many areas, such as semantic search and recommendation systems. Subsequently, most reasoning models are inherently transductive and ignore uncertainties of KGs, making it difficult to generalize to unseen entities. Moreover, existing approaches usually require each entity in the KG to have sufficient training samples, which leads to the overfitting of the entity having few instances. In fact, long-tail distributions are quite widespread in KGs, and newly emerging entities will tend to have only a few related triples. In this work, we aim at studying knowledge graph reasoning under a challenging setting where only limited training samples are available. Specifically, we propose a Bayesian inductive reasoning method and incorporate meta-learning techniques in few-shot learning to solve data deficiency and uncertainties. We design a Bayesian graph neural network as a meta-learner to achieve Bayesian inference, which can extrapolate meta-knowledge from observed KG to emerging entities. We conduct extensive experiments on two large-scale benchmark datasets, and the results demonstrate considerable performance improvement with the proposed approach over other baselines.

References

[1]

Jinheon Baek, Dong Bok Lee, and Sung Ju Hwang. 2020. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems. 546–560.

[2]

Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 5184–5193.

[3]

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural network. In Proceedings of the 32nd International Conference on Machine Learning. 1613–1622.

[4]

Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 1247–1250.

Digital Library

[5]

Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems. 2787–2795.

Digital Library

[6]

Jiarui Cai, Yizhou Wang, and Jenq-Neng Hwang. 2021. ACE: Ally complementary experts for solving long-tailed recognition in one-shot. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 112–121.

[7]

Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, and Qingming Huang. 2021. Dual quaternion knowledge graph embeddings. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 6894–6902.

[8]

Melisachew Wudage Chekol, Giuseppe Pirrò, Joerg Schoenfisch, and Heiner Stuckenschmidt. 2017. Marrying uncertainty and time in knowledge graphs. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 88–94.

[9]

Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. 2019. Meta relational learning for few-shot link prediction in knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 4216–4225.

[10]

Zhu-Mu Chen, Mi-Yen Yeh, and Tei-Wei Kuo. 2021. PASSLEAF: A Pool-bAsed Semi-Supervised LEArning Framework for uncertain knowledge graph embedding. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 4019–4026.

[11]

Siamak Zamani Dadaneh, Shahin Boluki, Mingyuan Zhou, and Xiaoning Qian. 2020. ARSM gradient estimator for supervised learning to rank. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 3157–3161.

[12]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems. 3837–3845.

[13]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 1811–1818.

[14]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning. 1126–1135.

Digital Library

[15]

Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Jimenez Rezende, and S. M. Ali Eslami. 2018. Conditional neural processes. In Proceedings of the 35th International Conference on Machine Learning. 1690–1699.

[16]

Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, and Yuji Matsumoto. 2017. Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1802–1808.

[17]

Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. 2015. Learning to represent knowledge graphs with Gaussian embedding. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management. 623–632.

Digital Library

[18]

Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. 687–696.

[19]

Taisong Jin, Huaqiang Dai, Liujuan Cao, Baochang Zhang, Feiyue Huang, Yue Gao, and Rongrong Ji. 2022. Deepwalk-aware graph convolutional networks. Science China Information Sciences 65, 5 (2022), 1–15.

[20]

Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations. 1–14.

[21]

Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In Proceedings of the CML Deep Learning Workshop. 1–8.

[22]

Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2018. Multi-hop knowledge graph reasoning with reward shaping. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 3243–3253.

[23]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2181–2187.

[24]

Jiaqi Ma, Weijing Tang, Ji Zhu, and Qiaozhu Mei. 2019. A flexible generative framework for graph-based semi-supervised learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems. 3276–3285.

[25]

Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2018. A simple neural attentive meta-learner. In Proceedings of the 6th International Conference on Learning Representations. 1–17.

[26]

Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. 2019. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 4710–4723.

[27]

Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Q. Phung. 2018. A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. 327–333.

[28]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning. 809–816.

Digital Library

[29]

Sarah Parisot, Pedro M. Esperança, Steven McDonagh, Tamas J. Madarasz, Yongxin Yang, and Zhenguo Li. 2022. Long-tail recognition via compositional knowledge transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6939–6948.

[30]

Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. 2019. Meta-learning with latent embedding optimization. In Proceedings of the 7th International Conference on Learning Representations. 1–17.

[31]

Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In The Semantic Web. Lecture Notes in Computer Science, Vol. 10843. Springer, 593–607.

[32]

Baoxu Shi and Tim Weninger. 2018. Open-world knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 1957–1964.

[33]

Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. ConceptNet 5.5: An open multilingual graph of general knowledge. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 4444–4451.

[34]

Jian Sun, Yu Zhou, and Chengqing Zong. 2022. One-shot relation learning for knowledge graphs via neighborhood aggregation and paths encoding. ACM Transactions on Asian and Low-Resource Language Information Processing 21, 3 (2022), 1–19.

Digital Library

[35]

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations. 1–18.

[36]

Yi-Lin Tuan, Sajjad Beygi, Maryam Fazel-Zarandi, Qiaozi Gao, Alessandra Cervone, and William Yang Wang. 2022. Towards large-scale interpretable knowledge graph reasoning for dialogue systems. In Findings of the Association for Computational Linguistics: ACL 2022. Association for Computational Linguistics, Dublin, Ireland, 383–395.

[37]

Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. 2020. Composition-based multi-relational graph convolutional networks. In Proceedings of the 8th International Conference on Learning Representations. 1–15.

[38]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems. 5998–6008.

[39]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations. 1–12.

[40]

Peifeng Wang, Jialong Han, Chenliang Li, and Rong Pan. 2019. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In Proceedings of the AAAI Conference on Artificial Intelligence. 7152–7159.

Digital Library

[41]

Xiting Wang, Kunpeng Liu, Dongjie Wang, Le Wu, Yanjie Fu, and Xing Xie. 2022. Multi-level recommendation reasoning over knowledge graphs with reinforcement learning. In Proceedings of the ACM Web Conference. 2098–2108.

Digital Library

[42]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112–1119.

[43]

Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 564–573.

[44]

Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2018. One-shot relational learning for knowledge graphs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 1980–1990.

[45]

Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representations. 1–12.

[46]

Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V. Chawla. 2020. Few-shot knowledge graph completion. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 3041–3048.

[47]

Ningyu Zhang, Shumin Deng, Zhanlin Sun, Jiaoyan Chen, Wei Zhang, and Huajun Chen. 2020. Relation adversarial network for low resource knowledge graph completion. In Proceedings of the Web Conference. 1–12.

Digital Library

[48]

Yingxue Zhang, Soumyasundar Pal, Mark Coates, and Deniz Üstebay. 2019. Bayesian graph convolutional neural networks for semi-supervised classification. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 5829–5836.

Digital Library

[49]

Zhaoli Zhang, Zhifei Li, Hai Liu, and Neal N. Xiong. 2022. Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering 34, 5 (2022), 2335–2347.

[50]

Yu Zhao, Anxiang Zhang, Ruobing Xie, Kang Liu, and Xiaojie Wang. 2020. Connecting embeddings for knowledge graph entity typing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6419–6428.

Cited By

Lu WWei SPeng XWang YNaseem UWang S(2024)Medical Question Summarization with Entity-driven Contrastive LearningACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365216023:4(1-19)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3652160

Index Terms

BayesKGR: Bayesian Few-Shot Learning for Knowledge Graph Reasoning
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Reasoning about belief and knowledge
    2. Natural language processing
  2. Machine learning

Recommendations

Few-Shot Knowledge Graph Entity Typing
Advances in Knowledge Discovery and Data Mining
Abstract
Knowledge graph entity typing, which is an important way to complete knowledge graphs (KGs), aims at predicting the associating type of certain given entities without any external knowledge. However, previous methods suppose that many (entity, ...
Few-shot Knowledge Reasoning Method based on Attention Mechanism
ICCPR '19: Proceedings of the 2019 8th International Conference on Computing and Pattern Recognition

As a core issue of knowledge graph research, knowledge graph reasoning and complementation technology have always been a hot topic of current research. Existing knowledge graph reasoning techniques usually requires a large amount of training for each ...
Prior-knowledge and attention based meta-learning for few-shot learning
Abstract
Recently, meta-learning has been shown to be a promising way to solve few-shot learning. In this paper, inspired by the human cognition process, which utilizes both prior-knowledge and visual attention when learning new knowledge, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 22, Issue 6

June 2023

635 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3604597

Editor:
Imed Zitouni
Google, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2023

Online AM: 27 March 2023

Accepted: 13 March 2023

Revised: 29 December 2022

Received: 23 June 2022

Published in TALLIP Volume 22, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
530
Total Downloads

Downloads (Last 12 months)259
Downloads (Last 6 weeks)32

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu WWei SPeng XWang YNaseem UWang S(2024)Medical Question Summarization with Entity-driven Contrastive LearningACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365216023:4(1-19)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3652160

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents