research-article

INK: knowledge graph embeddings for node classification

Authors:

Bram Steenwinckel,

Gilles Vandewiele,

Terencio Agozzino,

Filip De Turck,

Femke OngenaeAuthors Info & Claims

Data Mining and Knowledge Discovery, Volume 36, Issue 2

Pages 620 - 667

https://doi.org/10.1007/s10618-021-00806-z

Published: 01 March 2022 Publication History

Abstract

Deep learning techniques are increasingly being applied to solve various machine learning tasks that use Knowledge Graphs as input data. However, these techniques typically learn a latent representation for the entities of interest internally, which is then used to make decisions. This latent representation is often not comprehensible to humans, which is why deep learning techniques are often considered to be black boxes. In this paper, we present INK: Instance Neighbouring by using Knowledge, a novel technique to learn binary feature-based representations, which are comprehensible to humans, for nodes of interest in a knowledge graph. We demonstrate the predictive power of the node representations obtained through INK by feeding them to classical machine learning techniques and comparing their predictive performances for the node classification task to the current state of the art: Graph Convolutional Networks (R-GCN) and RDF2Vec. We perform this comparison both on benchmark datasets and using a real-world use case.

References

[1]

Anelli VW, Noia TD, Sciascio ED, Ragone A, Trotta J (2019) How to make latent factors interpretable by feeding factorization machines with knowledge graphs. arXiv:1909.05038

[2]

Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: a nucleus for a web of open data. In: The semantic web, pp 722–735. Springer

[3]

Baldassarre F, Azizpour H (2019) Explainability techniques for graph convolutional networks. arXiv preprint arXiv:1905.13686

[4]

Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ (eds) Advances in Neural information processing systems, vol. 26. Curran Associates, Inc

[5]

De Boer V, Wielemaker J, Van Gent J, Hildebrand M, Isaac A, Van Ossenbruggen J, Schreiber G (2012) Supporting linked data production for cultural heritage institutes: the Amsterdam museum case study. In: Extended semantic web conference, pp 733–747. Springer

[6]

Ehrlinger L and Wöß W Towards a definition of knowledge graphs SEMANTiCS (Posters, Demos, SuCCESS) 2016 48 1-4

[7]

Gulisano V, Jerzak Z, Katerinenko R, Strohbach M, Ziekow H (2017) The debs 2017 grand challenge. In: Proceedings of the 11th ACM international conference on distributed and event-based systems, DEBS ’17, p. 271-273. Association for Computing Machinery, New York, NY, USA.

[8]

Gunel B (2019) Robust relational graph convolutional networks

[9]

Hamilton WL, Ying R, Leskovec J (2017) Representation learning on graphs: methods and applications. arXiv preprint arXiv:1709.05584

[10]

Kazemi SM, Poole D (2018) Simple embedding for link prediction in knowledge graphs. arXiv preprint arXiv:1802.04868

[11]

Khalid S, Khalil T, Nasreen S (2014) A survey of feature selection and feature extraction techniques in machine learning. In: 2014 Science and information conference, pp 372–378. IEEE

[12]

Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

[13]

Krech D (2006) Rdflib: a python library for working with rdf

[14]

Lecue F On the role of knowledge graphs in explainable ai Semantic Web 2020 11 1 41-51

[15]

Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S (2015a) Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379

[16]

Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015b) Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI conference on artificial intelligence, vol. 29

[17]

Lösch U, Bloehdorn S, Rettinger A (2012) Graph kernels for rdf data. In: Extended semantic web conference, pp 134–148. Springer

[18]

Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems 30, pp 4765–4774. Curran Associates, Inc. http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf

[19]

Marzagao DK, Huynh TD, Helal A, Moreau L (2020) Provenance graph kernel. arXiv preprint arXiv:2010.10343

[20]

Mikolov T, Chen K, Corrado G, Dean J (2013a) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

[21]

Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013b) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

[22]

Miller E An introduction to the resource description framework Bull Am Soc Inf Sci Technol 1998 25 1 15-19

[23]

Neil D, Briody J, Lacoste A, Sim A, Creed P, Saffari A (2018) Interpretable graph convolutional neural networks for inference on noisy knowledge graphs. arXiv preprint arXiv:1812.00279

[24]

Nguyen DQ, Nguyen TD, Nguyen DQ, Phung D (2017) A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121

[25]

Nickel M, Tresp V, Kriegel HP (2011) A three-way model for collective learning on multi-relational data. In: ICML

[26]

Nickel M, Murphy K, Tresp V, and Gabrilovich E A review of relational machine learning for knowledge graphs Proc IEEE 2015 104 1 11-33

[27]

Paulheim H (2012) Generating possible interpretations for statistics from linked open data. In: Extended semantic web conference, pp 560–574. Springer

[28]

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al. Scikit-learn: machine learning in python J Mach Learn Res 2011 12 2825-2830

[29]

Portisch J, Hladik M, Paulheim H (2021) Finmatcher at finsim-2: hypernym detection in the financial services domain using knowledge graphs. arXiv preprint arXiv:2103.01576

[30]

Ristoski P, Paulheim H, Svátek V, Zeman V (2015) The linked data mining challenge 2015. In: KNOW@ LOD

[31]

Ristoski P, De Vries GKD, Paulheim H (2016a) A collection of benchmark datasets for systematic evaluations of machine learning on the semantic web. In: International semantic web conference, pp 186–194. Springer

[32]

Ristoski P, Paulheim H, Svátek V, Zeman V (2016b) The linked data mining challenge 2016. In: (KNOW@ LOD/CoDeS)@ ESWC

[33]

Ristoski P, Rosati J, Di Noia T, De Leone R, and Paulheim H Rdf2vec: Rdf graph embeddings and their applications Semantic Web 2019 10 4 721-752

[34]

Ristoski P, Gentile AL, Alba A, Gruhl D, and Welch S Large-scale relation extraction from web documents and knowledge graphs with human-in-the-loop J Web Semantics 2020 60 100546

[35]

Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. arXiv preprint arXiv:1710.09829

[36]

Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, pp 593–607. Springer

[37]

Tan Z, Zhao X, Fang Y, and Xiao W Gtrans: generic knowledge graph embedding via multi-state entities and dynamic relation spaces IEEE Access 2018 6 8232-8244

[38]

Taniar D, Rahayu JW (2006) Web semantics & ontology. Igi Global

[39]

Thanapalasingam T, van Berkel L, Bloem P, Groth P (2021) Relational graph convolutional networks: A closer look. arXiv preprint arXiv:2107.10015

[40]

Trouillon T, Welbl J, Riedel S, Gaussier E, Bouchard G (2016) Complex embeddings for simple link prediction. In: Proceedings of the 33rd international conference on international conference on machine learning - Volume 48, ICML’16, pp 2071–2080. JMLR.org

[41]

Union S (2018) Stardog

[42]

Vandewiele G, Steenwinckel B, Ongenae F, De Turck F (2019) Inducing a decision tree with discriminative paths to classify entities in a knowledge graph. In: SEPDA2019, the 4th International workshop on semantics-powered data mining and analytics, pp 1–6

[43]

Vandewiele G, Steenwinckel B, Agozzino T, Weyns M, Bonte P, Ongenae F, Turck FD (2020a) pyrdf2vec: Python implementation and extension of rdf2vec. IDLab. https://github.com/IBCNServices/pyRDF2Vec

[44]

Vandewiele G, Steenwinckel B, Bonte P, Weyns M, Paulheim H, Ristoski P, De Turck F, Ongenae F (2020b) Walk extraction strategies for node embeddings with rdf2vec in knowledge graphs. arXiv preprint arXiv:2009.04404

[45]

Voit MM, Paulheim H (2021) Bias in knowledge graphs—an empirical study with movie recommendation and different language editions of dbpedia. arXiv preprint arXiv:2105.00674

[46]

Vrandečić D and Krötzsch M Wikidata: a free collaborative knowledgebase Commun ACM 2014 57 10 78-85

[47]

Vu T, Nguyen TD, Nguyen DQ, Phung D, et al. (2019) A capsule network-based embedding model for knowledge graph completion and search personalization. In: Proceedings of the 2019 conference of the North American Chapter of the Association for computational linguistics: human language technologies, Volume 1 (Long and Short Papers), pp 2180–2189

[48]

Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI conference on artificial intelligence, vol. 28

[49]

Wang M, Zheng D, Ye Z, Gan Q, Li M, Song X, Zhou J, Ma C, Yu L, Gai Y, Xiao T, He T, Karypis G, Li J, Zhang Z (2019) Deep graph library: a graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315

[50]

Wilcke X, Bloem P, and De Boer V The knowledge graph as the default data model for learning on heterogeneous knowledge Data Sci 2017 1 1–2 39-57

[51]

Xiao H, Huang M, Hao Y, Zhu X (2015) Transg: a generative mixture model for knowledge graph embedding. arXiv preprint arXiv:1509.05488

[52]

Yanardag P, Vishwanathan S (2015) Deep graph kernels. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1365–1374

[53]

Yang B, Yih WT, He X, Gao J, Deng L (2014) Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575

[54]

Zhang Z, Cao L, Chen X, Tang W, Xu Z, and Meng Y Representation learning of knowledge graphs with entity attributes IEEE Access 2020 8 7435-7441

[55]

Zouaq A, Martel F (2020) What is the schema of your knowledge graph? leveraging knowledge graph embeddings and clustering for expressive taxonomy learning. In: Proceedings of the international workshop on semantic big data, pp 1–6

Cited By

Toroghi ASanner SWooldridge MDy JNatarajan S(2024)Bayesian inference with complex knowledge graph evidenceProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i18.30040(20550-20558)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i18.30040
Steenwinckel BSoete CMoens PMussche JHoecke SOngenae F(2024)Quality in Color: Using Knowledge Graphs for Enhanced Quality Control in an Automotive PaintshopThe Semantic Web – ISWC 202410.1007/978-3-031-77847-6_13(236-252)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1007/978-3-031-77847-6_13
Tailhardat LTroncy RChabot Y(2023)Leveraging Knowledge Graphs For Classifying Incident Situations in ICT SystemsProceedings of the 18th International Conference on Availability, Reliability and Security10.1145/3600160.3604991(1-9)Online publication date: 29-Aug-2023
https://dl.acm.org/doi/10.1145/3600160.3604991
Show More Cited By

Recommendations

A Knowledge Graph Embedding Model Based on Node-Relation Fusion Perception
ICNCC '20: Proceedings of the 2020 9th International Conference on Networks, Communication and Computing

Although modeling graph structure data with graph convolution network has become a hot research topic in recent years, the research subject in most papers are simple undirected graphs while the multi-relational ones, for example, knowledge graphs, of ...
OpenWGL: open-world graph learning for unseen class node classification
Abstract
Graph learning, such as node classification, is typically carried out in a closed-world setting. A number of nodes are labeled, and the learning goal is to correctly classify remaining (unlabeled) nodes into classes, represented by the labeled ...
Node Similarity Preserving Graph Convolutional Network Based on Full-frequency Information for Node Classification
Abstract
Recently, graph neural networks have achieved good performance in graph representation learning. However, most graph neural networks only utilize node low-frequency signals and destroy node similarity when aggregating graph structure and node ...

Comments

Information & Contributors

Information

Published In

cover image Data Mining and Knowledge Discovery

Data Mining and Knowledge Discovery Volume 36, Issue 2

Mar 2022

403 pages

ISSN:1384-5810

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2021.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 March 2022

Accepted: 09 October 2021

Received: 31 January 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Toroghi ASanner SWooldridge MDy JNatarajan S(2024)Bayesian inference with complex knowledge graph evidenceProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i18.30040(20550-20558)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i18.30040
Steenwinckel BSoete CMoens PMussche JHoecke SOngenae F(2024)Quality in Color: Using Knowledge Graphs for Enhanced Quality Control in an Automotive PaintshopThe Semantic Web – ISWC 202410.1007/978-3-031-77847-6_13(236-252)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1007/978-3-031-77847-6_13
Tailhardat LTroncy RChabot Y(2023)Leveraging Knowledge Graphs For Classifying Incident Situations in ICT SystemsProceedings of the 18th International Conference on Availability, Reliability and Security10.1145/3600160.3604991(1-9)Online publication date: 29-Aug-2023
https://dl.acm.org/doi/10.1145/3600160.3604991
Verkijk SRoothaert RPernisch RSchlobach S(2023)Do you catch my drift? On the usage of embedding methods to measure concept shift in knowledge graphsProceedings of the 12th Knowledge Capture Conference 202310.1145/3587259.3627555(70-74)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3587259.3627555
Schramm SWehner CSchmid U(2023)Comprehensible Artificial Intelligence on Knowledge GraphsWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2023.10080679:COnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.websem.2023.100806
Ismaeil YStepanova DTran TBlockeel H(2023)FeaBI: A Feature Selection-Based Framework for Interpreting KG EmbeddingsThe Semantic Web – ISWC 202310.1007/978-3-031-47240-4_32(599-617)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-47240-4_32
Steenwinckel BVandewiele GAgozzino TOngenae F(2023)pyRDF2Vec: A Python Implementation and Extension of RDF2VecThe Semantic Web10.1007/978-3-031-33455-9_28(471-483)Online publication date: 28-May-2023
https://dl.acm.org/doi/10.1007/978-3-031-33455-9_28

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents