Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Knowledge Graph Embedding for Link Prediction: A Comparative Analysis

Published: 04 January 2021 Publication History

Abstract

Knowledge Graphs (KGs) have found many applications in industrial and in academic settings, which in turn, have motivated considerable research efforts towards large-scale information extraction from a variety of sources. Despite such efforts, it is well known that even the largest KGs suffer from incompleteness; Link Prediction (LP) techniques address this issue by identifying missing facts among entities already in the KG. Among the recent LP techniques, those based on KG embeddings have achieved very promising performance in some benchmarks. Despite the fast-growing literature on the subject, insufficient attention has been paid to the effect of the design choices in those methods. Moreover, the standard practice in this area is to report accuracy by aggregating over a large number of test facts in which some entities are vastly more represented than others; this allows LP methods to exhibit good results by just attending to structural properties that include such entities, while ignoring the remaining majority of the KG. This analysis provides a comprehensive comparison of embedding-based LP methods, extending the dimensions of analysis beyond what is commonly available in the literature. We experimentally compare the effectiveness and efficiency of 18 state-of-the-art methods, consider a rule-based baseline, and report detailed analysis over the most popular benchmarks in the literature.

References

[1]
Naser Ahmadi, Viet-Phi Huynh, Venkata Vamsikrishna Meduri, Stefano Ortona, and Paolo Papotti. 2020. Mining expressive rules in knowledge graphs. Journal of Data and Information Quality 12, 2 (2020), 8:1--8:27.
[2]
Farahnaz Akrami, Lingbing Guo, Wei Hu, and Chengkai Li. 2018. Re-evaluating embedding-based knowledge graph completion methods. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management.
[3]
Bo An, Bo Chen, Xianpei Han, and Le Sun. 2018. Accurate text-enhanced knowledge graph representation learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
[4]
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The Semantic Web. Springer.
[5]
Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.
[6]
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of Data.
[7]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems.
[8]
Hongyun Cai, Vincent W. Zheng, and Kevin Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques and applications. IEEE Transactions on Knowledge and Data Engineering 30, 9 (2018), 1616--1637.
[9]
Chandrahas, Aditya Sharma, and Partha P. Talukdar. 2018. Towards understanding the geometry of knowledge graph embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.
[10]
Luca Costabello, Sumit Pai, Chan Le Van, Rory McGrath, Nicholas McCarthy, and Pedro Tabacof. 2019. AmpliGraph: A Library for Representation Learning on Knowledge Graphs. Retrieved October 10, 2019 from
[11]
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.
[12]
Xin Luna Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
[13]
Takuma Ebisu and Ryutaro Ichise. 2018. TorusE: Knowledge graph embedding on a lie group. In Proceedings of the 2018 AAAI Conference on Artificial Intelligence.
[14]
Bradley Efron and Robert J. Tibshirani. 1994. An Introduction to the Bootstrap. CRC Press.
[15]
Facebook Research. [n.d.]. Knowledge Base Completion (kbc). Retrieved October 10, 2019 from https://github.com/facebookresearch/kbc.
[16]
Bahare Fatemi. [n.d.]. A faster SimplE implementation. Retrieved October 10, 2019 from https://github.com/baharefatemi/SimplE.
[17]
Bahare Fatemi, Perouz Taslakian, David Vázquez, and David Poole. 2019. Knowledge hypergraphs: Extending knowledge graphs beyond binary relations. CoRR abs/1906.00137 (2019).
[18]
Luis Antonio Galárraga, Christina Teflioudi, Katja Hose, and Fabian Suchanek. 2013. AMIE: Association rule mining under incomplete evidence in ontological knowledge bases. In Proceedings of the 22nd International Conference on World Wide Web.
[19]
Luis Antonio Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2015. Fast rule mining in ontological knowledge bases with AMIE+. VLDB Journal 24, 6 (2015), 707--730.
[20]
Genet Asefa Gesese, Russa Biswas, and Harald Sack. 2019. A comprehensive survey of knowledge graph embeddings with literals: Techniques and applications. In Proceedings of the 2019 Workshop on Deep Learning for Knowledge Graphs.
[21]
Lingbing Guo, Zequn Sun, and Wei Hu. 2019. Learning to exploit long-term relational dependencies in knowledge graphs. In Proceedings of the 2019 International Conference on Machine Learning.
[22]
Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. 2018. Knowledge graph embedding with iterative guidance from soft rules. In Proceedings of the 2018 AAAI Conference on Artificial Intelligence.
[23]
Víctor Gutiérrez-Basulto and Steven Schockaert. 2018. From knowledge graph embedding to ontology embedding? An analysis of the compatibility between vector space representations and rules. In Proceedings of the 2018 International Conference on Principles of Knowledge Representation and Reasoning.
[24]
Katsuhiko Hayashi and Masashi Shimbo. 2017. On the equivalence of holographic and complex embeddings for link prediction. In Proceedings of the 2017 Annual Meeting of the Association for Computational Linguistics.
[25]
Frank L. Hitchcock. 1927. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics 6, 1--4 (1927), 164--189.
[26]
John Hopfield. 1982. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences 79, 8 (1982), 2554--2558.
[27]
Eduard Hovy, Roberto Navigli, and Simone Paolo Ponzetto. 2013. Collaboratively built semi-structured content and artificial intelligence: The story so far. Artificial Intelligence 194, 1 (2013), 2--27.
[28]
Viet-Phi Huynh and Paolo Papotti. 2019. A benchmark for fact checking algorithms built on knowledge bases. In Proceedings of the 2019 ACM International Conference on Information and Knowledge Management.
[29]
Xiaotian Jiang, Quan Wang, and Bin Wang. 2019. Adaptive convolution for multi-relational learning. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
[30]
Rudolf Kadlec, Ondrej Bajgar, and Jan Kleindienst. 2017. Knowledge base completion: Baselines strike back. In Proceedings of the 2nd Workshop on Representation Learning for NLP.
[31]
Seyed Mehran Kazemi and David Poole. 2018. SimplE embedding for link prediction in knowledge graphs. In Proceedings of the 2018 International Conference on Neural Information Processing Systems.
[32]
Tamara G. Kolda and Brett W. Bader. 2009. Tensor decompositions and applications. SIAM Review 51, 3 (2009), 455--500.
[33]
Simeon Kostadinov. 2018. Recurrent Neural Networks with Python Quick Start Guide: Sequential Learning and Language Modeling with TensorFlow. Packt Publishing Ltd.
[34]
Timothée Lacroix, Nicolas Usunier, and Guillaume Obozinski. 2018. Canonical tensor decomposition for knowledge base completion. In Proceedings of the 2018 International Conference on Machine Learning.
[35]
Ni Lao and William W. Cohen. 2010. Relational retrieval using a combination of path-constrained random walks. Machine Learning 81, 1 (2010), 53--67.
[36]
Ni Lao, Tom Mitchell, and William W. Cohen. 2011. Random walk inference and learning in a large scale knowledge base. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing.
[37]
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (1998), 2278--2324.
[38]
Yankai Lin, Zhiyuan Liu, Huan-Bo Luan, Maosong Sun, Siwei Rao, and Song Liu. 2015. Modeling relation paths for representation learning of knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
[39]
Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the 2017 International Conference on Machine Learning.
[40]
Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2015. YAGO3: A knowledge base from multilingual Wikipedias. In Proceedings of the 7th Biennial Conference on Innovative Data Systems Research.
[41]
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press.
[42]
Christian Meilicke, Manuel Fink, Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, and Heiner Stuckenschmidt. 2018. Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion. In Proceedings of the 17th International Semantic Web Conference.
[43]
Christian Meilicke, Melisachew Wudage Chekol, Daniel Ruffinelli, and Heiner Stuckenschmidt. 2019. Anytime bottom-up rule learning for knowledge graph completion. In Proceedings of the 28th International Joint Conference on Artificial Intelligence.
[44]
Christian Meilicke, Melisachew Wudage Chekol, Manuel Fink, and Heiner Stuckenschmidt. 2020. Reinforced anytime bottom up rule learning for knowledge graph completion. arXiv preprint arXiv:2004.04412 (2020).
[45]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the 2013 International Conference on Learning Representations.
[46]
Sameh K. Mohamed, Vít Novácek, Pierre-Yves Vandenbussche, and Emir Muñoz. 2019. Loss functions in knowledge graph embedding models. In Proceedings of the 2019 Workshop on Deep Learning for Knowledge Graphs.
[47]
Dat Quoc Nguyen. 2017. An overview of embedding models of entities and relationships for knowledge base completion. CoRR abs/1703.08098 (2017). arxiv:1703.08098
[48]
Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. 2016. STransE: A novel embedding model of entities and relationships in knowledge bases. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
[49]
Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Q. Phung. 2018. A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
[50]
Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Q. Phung. 2019. A capsule network-based embedding model for knowledge graph completion and search personalization. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
[51]
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 2011 International Conference on Machine Learning.
[52]
Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2015. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE 104, 1 (2015), 11--33.
[53]
Maximilian Nickel, Lorenzo Rosasco, and Tomaso A. Poggio. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the 2016 AAAI Conference on Artificial Intelligence.
[54]
Heiko Paulheim. 2017. Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic Web 8, 3 (2017), 489--508.
[55]
Pouya Pezeshkpour, Liyan Chen, and Sameer Singh. 2018. Embedding multimodal relational data for knowledge base completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
[56]
Pouya Pezeshkpour, Yifan Tian, and Sameer Singh. 2019. Investigating robustness and interpretability of link prediction via adversarial modifications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
[57]
Andrea Rossi and Antonio Matinata. 2020. Knowledge graph embeddings: Are relation-learning models learning relations? In Proceedings of the EDBT/ICDT 2020 Joint Conference.
[58]
Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. 2017. Dynamic routing between capsules. In Proceedings of the 2017 International Conference on Neural Information Processing Systems.
[59]
Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A core of semantic knowledge. In Proceedings of the 2007 International Conference on World Wide Web. ACM.
[60]
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the International Conference on Learning Representations.
[61]
Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality.
[62]
Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
[63]
Hung Nghiep Tran and Atsuhiro Takasu. 2019. Analyzing knowledge graph embedding methods from a multi-embedding interaction perspective. In Proceedings of the EDBT/ICDT 2019 Joint Conference.
[64]
Bayu Distiawan Trisedya, Jianzhong Qi, and Rui Zhang. 2019. Entity alignment between knowledge graphs using attribute embeddings. In Proceedings of the 2019 AAAI Conference on Artificial Intelligence.
[65]
Rakshit Trivedi, Bunyamin Sisman, Xin Luna Dong, Christos Faloutsos, Jun Ma, and Hongyuan Zha. 2018. LinkNBed: Multi-graph representation learning with entity linkage. In Proceedings of the 2018 Annual Meeting of the Association for Computational Linguistics.
[66]
Théo Trouillon and Maximilian Nickel. 2017. Complex and holographic embeddings of knowledge graphs: A comparison. CoRR abs/1707.01475 (2017).
[67]
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 2016 International Conference on Machine Learning.
[68]
University Mannheim, Data and Web Science Group. [n.d.]. AnyBURL. Retrieved October 10, 2019 from http://web.informatik.uni-mannheim.de/AnyBURL/.
[69]
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, Nilesh Agrawal, and Partha P. Talukdar. 2020. InteractE: Improving convolution-based knowledge graph embeddings by increasing feature interactions. In Proceedings of the 2020 AAAI Conference on Artificial Intelligence.
[70]
Ellen M. Voorhees. 1999. The TREC-8 question answering track report. In Proceedings of the Text REtrieval Conference.
[71]
Denny Vrandecic and Markus Krötzsch. 2014. Wikidata: A free collaborative knowledge base. Communications of the ACM 57, 10 (2014), 78--85.
[72]
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29, 12 (2017), 2724--2743.
[73]
Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, Samuel Broscheit, and Christian Meilicke. 2019. On evaluating embedding models for knowledge base completion. In Proceedings of the 4th Workshop on Representation Learning for NLP.
[74]
Zhigang Wang and Juan-Zi Li. 2016. Text-enhanced representation learning for knowledge graph. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence.
[75]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 2014 AAAI Conference on Artificial Intelligence.
[76]
Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, and Richong Zhang. 2016. On the representation and embedding of knowledge bases beyond binary relations. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence.
[77]
Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and Dekang Lin. 2014. Knowledge base completion via search-based question answering. In Proceedings of the 2014 International Conference on World Wide Web.
[78]
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 2015 International Conference on Learning Representations.
[79]
Wen Zhang, Bibek Paudel, Wei Zhang, Abraham Bernstein, and Huajun Chen. 2019. Interaction embeddings for prediction and explanation in knowledge graphs. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining.
[80]
Yongqi Zhang, Quanming Yao, Wenyuan Dai, and Lei Chen. 2020. AutoSF: Searching scoring functions for knowledge graph embedding. In Proceedings of the 2020 IEEE 36th International Conference on Data Engineering. IEEE.
[81]
Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proceedings of the 2020 AAAI Conference on Artificial Intelligence.

Cited By

View all
  • (2025)Expressiveness Analysis and Enhancing Framework for Geometric Knowledge Graph Embedding ModelsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348691537:1(306-318)Online publication date: 1-Jan-2025
  • (2025)DuAK: Reinforcement Learning-Based Knowledge Graph Reasoning for Steel Surface Defect DetectionIEEE Transactions on Automation Science and Engineering10.1109/TASE.2023.330758822(557-569)Online publication date: 2025
  • (2025)Line graph neural networks for link weight predictionPhysica A: Statistical Mechanics and its Applications10.1016/j.physa.2025.130406661(130406)Online publication date: Mar-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 15, Issue 2
Survey Paper and Regular Papers
April 2021
524 pages
ISSN:1556-4681
EISSN:1556-472X
DOI:10.1145/3446665
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 January 2021
Accepted: 01 September 2020
Revised: 01 July 2020
Received: 01 January 2020
Published in TKDD Volume 15, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Knowledge graphs
  2. comparative analysis
  3. knowledge graph embeddings
  4. link prediction

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)933
  • Downloads (Last 6 weeks)74
Reflects downloads up to 01 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Expressiveness Analysis and Enhancing Framework for Geometric Knowledge Graph Embedding ModelsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348691537:1(306-318)Online publication date: 1-Jan-2025
  • (2025)DuAK: Reinforcement Learning-Based Knowledge Graph Reasoning for Steel Surface Defect DetectionIEEE Transactions on Automation Science and Engineering10.1109/TASE.2023.330758822(557-569)Online publication date: 2025
  • (2025)Line graph neural networks for link weight predictionPhysica A: Statistical Mechanics and its Applications10.1016/j.physa.2025.130406661(130406)Online publication date: Mar-2025
  • (2025)Uncertainty modeling for inductive knowledge graph embeddingNeural Networks10.1016/j.neunet.2024.107103184(107103)Online publication date: Apr-2025
  • (2025)The joint extraction of fact-condition statement and super relation in scientific text with table filling methodInformation Processing & Management10.1016/j.ipm.2024.10390662:1(103906)Online publication date: Jan-2025
  • (2025)Open knowledge graph completion with negative-aware representation learning and multi-source reliability inferenceInformation Fusion10.1016/j.inffus.2024.102729115(102729)Online publication date: Mar-2025
  • (2025)M2KGRL: A semantic-matching based framework for multimodal knowledge graph representation learningExpert Systems with Applications10.1016/j.eswa.2025.126388269(126388)Online publication date: Apr-2025
  • (2025)FTPComplEx: A flexible time perspective approach to temporal knowledge graph completionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109717139(109717)Online publication date: Jan-2025
  • (2025)Knowledge graph representation learning: A comprehensive and experimental overviewComputer Science Review10.1016/j.cosrev.2024.10071656(100716)Online publication date: May-2025
  • (2025)Simplified multi-view graph neural network for multilingual knowledge graph completionFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-024-3577-319:7Online publication date: 1-Jul-2025
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media