research-article

Multisource hierarchical neural network for knowledge graph embedding

Authors:

Juan YangAuthors Info & Claims

Volume 237, Issue PB

https://doi.org/10.1016/j.eswa.2023.121446

Published: 01 March 2024 Publication History

Abstract

Link prediction for knowledge graphs aims to obtain missing nodes in triples. In recent years, link prediction methods have made specific achievements in knowledge graph embedding. However, knowledge graphs are characteristic of the heterogeneity of multiple types of entities and relations. A vital issue is efficiently extracting complex graph information and constructing a knowledge-semantic fusion of multiple features. To overcome these issues, a novel link prediction framework based on a multisource hierarchical neural network for knowledge graph embedding (MSHE) is proposed. In particular, mapping functions obtain entities and relations from low- to high-dimensional mapping sources. The combination of mapping sources and entity-relation sources constitutes multisource knowledge information, which facilitates the integration of complex heterogeneous entities and relations. Unlike training a single independent network, the hierarchical embedding network proposed in this paper accumulates feature information at multiple levels. Then, to fuse feature information, our Highway multilayer perceptron (MLP) inductively generates high-quality knowledge information. Through extensive experiments, MSHE’s knowledge graph embedding outperformed the state-of-the-art baselines on FB15k-237 and YAGO3-10. Furthermore, MSHE achieves a Hits@10 score that is 3.8% and 2.7% higher than that of ComplexGCN on FB15K-237 and WN18RR, respectively. MSHE achieves a higher score in Hits@1 than DCN 10.0% in the dataset YAGO3-10. The experiments show that the MSHE achieved excellent results in the four datasets of comparative experiments.

References

[1]

Balazevic, I., Allen, C., & Hospedales, T. M. (2019). TuckER: Tensor Factorization for Knowledge Graph Completion. In Proceedings of the conference on 9th EMNLP-IJCNLP (pp. 5184–5193).

[2]

Bertram N., Dunkel J., Hermoso R., I am all EARS: Using open data and knowledge graph embeddings for music recommendations, Expert Systems with Applications 229 (Part A) (2023).

[3]

Bordes, A., Usunier, N., García-Durán, A., Weston, J., & Yakhnenko, O. (2013). Translating Embeddings for Modeling Multi-relational Data. In Proceedings of the conference on 27th NIPS (pp. 2787–2795).

[4]

Cai X., Xie L., Tian R., Cui Z., Explicable recommendation based on knowledge graph, Expert Systems with Applications 200 (2022).

Digital Library

[5]

Che, F., Zhang, D., Tao, J., Niu, M., & Zhao, B. (2020). ParamE: Regarding Neural Network Parameters as Relation Embeddings for Knowledge Graph Completion. In Proceedings of the conference on 34th AAAI, the 32nd IAAI and the 10th AAAI (pp. 2774–2781).

[6]

Chen, Z., Liao, J., & Zhao, X. (2023). Multi-granularity Temporal Question Answering over Knowledge Graphs. In Proceedings of the conference on ACL (pp. 11378–11392).

[7]

Chen, S., Liu, X., Gao, J., Jiao, J., Zhang, R., & Ji, Y. (2021). HittER: Hierarchical transformers for knowledge graph embeddings. In Proceedings of the conference on EMNLP (pp. 10395–10407).

[8]

Cui, Y., Wang, Y., Sun, Z., Liu, W., Jiang, Y., Han, K., & Hu, W. (2023). Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs. In Proceedings of the conference on 37th AAAI, the 35th IAAI, the 13th EAAI (pp. 4217–4224).

[9]

Dettmers, T., Minervini, P., Stenetorp, P., & Riedel, S. (2018). Convolutional 2D Knowledge Graph Embeddings. In Proceedings of the conference on 32nd AAAI, the 30th (IAAI-18), and the 8th AAAI (pp. 1811–1818).

[10]

Ebisu T., Ichise R., Generalized translation-based embedding of knowledge graph, IEEE Transactions on Knowledge and Data Engineering 32 (5) (2020) 941–951.

[11]

Fang H., Wang Y., Tian Z., Ye Y., Learning knowledge graph embedding with a dual-attention embedding network, Expert Systems with Applications 212 (2023).

[12]

Fu, G., Meng, Z., Han, Z., Ding, Z., Ma, Y., Schubert, M., Tresp, V., & Wattenhofer, R. (2022). TempCaps: A Capsule Network-based Embedding Model for Temporal Knowledge Graph Completion. In Proceedings of the conference on ACL (pp. 22–31).

[13]

Guan S., Cheng X., Bai L., Zhang F., Li Z., Zeng Y., Jin X., Guo J., What is event knowledge graph: A survey, IEEE Transactions on Knowledge and Data Engineering 35 (7) (2023) 7569–7589.

[14]

Guo L., Sun Z., Hu W., Learning to exploit long-term relational dependencies in knowledge graphs, in: Proceedings of the conference on 36th ICML. Vol. 97, PMLR, 2019, pp. 2505–2514.

[15]

Hayashi K., Kishimoto K., Shimbo M., Binarized embeddings for fast, space-efficient knowledge graph completion, IEEE Transactions on Knowledge and Data Engineering (2021) 1–12.

[16]

He, Y., Zhang, P., Liu, L., Liang, Q., Zhang, W., & Zhang, C. (2021). HIP Network: Historical Information Passing Network for Extrapolation Reasoning on Temporal Knowledge Graph. In Proceedings of the conference on 30th IJCAI (pp. 1915–1921).

[17]

Lai Y., Chen C., Zheng Z., Zhang Y., Block term decomposition with distinct time granularities for temporal knowledge graph completion, Expert Systems with Applications 201 (2022).

[18]

Li Z., Ji J., Fu Z., Ge Y., Xu S., Chen C., Zhang Y., Efficient non-sampling knowledge graph embedding, in: Proceedings of the conference on WWW, ACM / IW3C2, 2021, pp. 1727–1736.

[19]

Li Y., Liu L., Wang G., Du Y., Chen P., EGNN: Constructing explainable graph neural networks via knowledge distillation, Knowledge-Based Systems 241 (2022).

[20]

Li Z., Liu H., Zhang Z., Liu T., Xiong N.N., Learning knowledge graph embedding with heterogeneous relation attention networks, IEEE Transactions on Neural Networks and Learning Systems 33 (8) (2022) 3961–3973.

[21]

Mahdisoltani, F., Biega, J., & Suchanek, F. M. (2015). YAGO3: A Knowledge Base from Multilingual Wikipedias. In Proceedings of the conference on 7th CIDR (pp. 1–11).

[22]

Mezni H., Benslimane D., Bellatreche L., Context-aware service recommendation based on knowledge graph embedding, IEEE Transactions on Knowledge and Data Engineering (2021) 1.

[23]

Miller G.A., WordNet: A lexical database for English, Communications of the ACM 38 (11) (1995) 39–41.

Digital Library

[24]

Nguyen, D. Q., Nguyen, T. D., Nguyen, D. Q., & Phung, D. Q. (2018). A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In Proceedings of the conference on NAACL-HLT (pp. 327–333).

[25]

Nickel, M., Tresp, V., & Kriegel, H. (2011). A Three-Way Model for Collective Learning on Multi-Relational Data. In Proceedings of the conference on 28th ICML (pp. 809–816).

[26]

Schlichtkrull, M. S., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018). Modeling Relational Data with Graph Convolutional Networks. In Proceedings of the conference on 15th ESWC. Vol. 10843 (pp. 593–607).

[27]

Shang, C., Tang, Y., Huang, J., Bi, J., He, X., & Zhou, B. (2019). End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion. In Proceedings of the conference on 33rd AAAI, the 31st IAAI, the 9th AAAI and EAAI (pp. 3060–3067).

[28]

Song D., Zhang F., Lu M., Yang S., Huang H., DTransE: Distributed translating embedding for knowledge graph, IEEE Transactions on Parallel and Distributed Systems 32 (10) (2021) 2509–2523.

[29]

Su X., You Z., Huang D., Wang L., Wong L., Ji B., Zhao B., Biomedical knowledge graph embedding with capsule network for multi-label drug-drug interaction prediction, IEEE Transactions on Knowledge and Data Engineering 35 (6) (2023) 5640–5651.

[30]

Toutanova, K., & Chen, D. (2015). Observed Versus Latent Features for Knowledge Base and Text Inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality (pp. 57–66).

[31]

Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., & Bouchard, G. (2016). Complex Embeddings for Simple Link Prediction. In Proceedings of the conference on 33nd ICML. Vol. 48 (pp. 2071–2080).

[32]

Vashishth, S., Sanyal, S., Nitin, V., Agrawal, N., & Talukdar, P. P. (2020). InteractE: Improving Convolution-Based Knowledge Graph Embeddings by Increasing Feature Interactions. In Proceedings of the conference on 34th AAAI, the 32nd IAAI, the 10th AAAI and EAAI (pp. 3009–3016).

[33]

Vashishth, S., Sanyal, S., Nitin, V., & Talukdar, P. P. (2020). Composition-based Multi-Relational Graph Convolutional Networks. In Proceedings of the conference on 8th ICLR (pp. 1–16).

[34]

Wang X., Lyu S., Wang X., Wu X., Chen H., Temporal knowledge graph embedding via sparse transfer matrix, Information Sciences 623 (2023) 56–69.

[35]

Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014). Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the conference on 28th AAAI (pp. 1112–1119).

[36]

Xu, C., & Li, R. (2019). Relation Embedding with Dihedral Group in Knowledge Graph. In Proceedings of the conference on 57th ACL (pp. 263–272).

[37]

Yang, Z., Huang, Y., & Feng, J. (2023). Learning to Leverage High-Order Medical Knowledge Graph for Joint Entity and Relation Extraction. In Proceedings of the conference on ACL (pp. 9023–9035).

[38]

Yang, S., Tian, J., Zhang, H., Yan, J., He, H., & Jin, Y. (2019). TransMS: Knowledge Graph Embedding for Complex Relations by Multidirectional Semantics. In Proceedings of the conference on 28th IJCAI (pp. 1935–1942).

[39]

Yang, B., Yih, W., He, X., Gao, J., & Deng, L. (2015). Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Proceedings of the conference on 3rd ICLR (pp. 1–12).

[40]

Yu D., Yang Y., Zhang R., Wu Y., Knowledge embedding based graph convolutional network, in: Proceedings of the conference on WWW: The web conference, ACM / IW3C2, 2021, pp. 1619–1628.

[41]

Zeb A., Saif S., Chen J., Haq A.U., Gong Z., Zhang D., Complex graph convolutional network for link prediction in knowledge graphs, Expert Systems with Applications 200 (2022).

[42]

Zhang, Z., Cai, J., Zhang, Y., & Wang, J. (2020). Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction. In Proceedings of the conference on 34th AAAI, the 32nd IAAI, the 10th AAAI and EAAI (pp. 3065–3072).

[43]

Zhang Z., Huang J., Tan Q., Association rules enhanced knowledge graph attention network, Knowledge-Based Systems 239 (2022).

[44]

Zhang Z., Li Z., Liu H., Xiong N.N., Multi-scale dynamic convolutional network for knowledge graph embedding, IEEE Transactions on Knowledge and Data Engineering 34 (5) (2022) 2335–2347.

[45]

Zhang, F., Wang, X., Li, Z., & Li, J. (2020). TransRHS: A Representation Learning Method for Knowledge Graphs with Relation Hierarchical Structure. In Proceedings of the conference on 29th IJCAI (pp. 2987–2993).

[46]

Zhao, Y., Zhou, H., Xie, R., Zhuang, F., Li, Q., & Liu, J. (2021). Incorporating Global Information in Local Attention for Knowledge Representation Learning. In Proceedings of the conference on ACL, ACL/IJCNLP 2021 (pp. 1341–1351).

[47]

Zhou X., Niu L., Zhu Q., Zhu X., Liu P., Tan J., Guo L., Knowledge graph embedding by double limit scoring loss, IEEE Transactions on Knowledge and Data Engineering 14 (8) (2021) 1–14,.

[48]

Zhu, C., Chen, M., Fan, C., Cheng, G., & Zhang, Y. (2021). Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks. In Proceedings of the conference on 35th AAAI, the 33rd IAAI and the 11th EAAI 2021 (pp. 4732–4740).

[49]

Zhu Q., Zhou X., Tan J., Guo L., Knowledge base reasoning with convolutional-based recurrent neural networks, IEEE Transactions on Knowledge and Data Engineering 33 (5) (2021) 2015–2028.

Cited By

Zhang SLiang X(2024)An radicals construction technique based on dual quaternions and hierarchical transformersNeurocomputing10.1016/j.neucom.2024.128315604:COnline publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128315
Zhang CLi WMo YTang WLi HZeng Z(2024)BHRAM: a knowledge graph embedding model based on bidirectional and heterogeneous relational attention mechanismApplied Intelligence10.1007/s10489-024-06212-455:4Online publication date: 30-Dec-2024
https://dl.acm.org/doi/10.1007/s10489-024-06212-4
Liang QWang WYu JBao F(2024)Effective Knowledge Graph Embedding with Quaternion Convolutional NetworksNatural Language Processing and Chinese Computing10.1007/978-981-97-9437-9_15(183-196)Online publication date: 2-Nov-2024
https://dl.acm.org/doi/10.1007/978-981-97-9437-9_15
Show More Cited By

Index Terms

Multisource hierarchical neural network for knowledge graph embedding
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Kernel multi-attention neural network for knowledge graph embedding
Abstract
Link prediction is the problem of predicting missing link between entities and relations for knowledge graph. In recent years, some tasks have achieved great success for link prediction, but these tasks are far from expanding entity ...
Knowledge graph embedding with entity attributes using hypergraph neural networks

Knowledge graph embedding is aimed at capturing the semantic information of entities by modeling the structural information between entities. For long-tail entities which lack sufficient structural information, general knowledge graph embedding models ...
Multiview feature augmented neural network for knowledge graph embedding
Abstract
Link prediction in knowledge graph embedding is a meaningful research topic. Knowledge graph embedding (KGE) focuses on the problem of predicting missing links based on triples. Neural networks have been the paradigm of choice in ...

Comments

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 237, Issue PB

Mar 2024

1582 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 March 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang SLiang X(2024)An radicals construction technique based on dual quaternions and hierarchical transformersNeurocomputing10.1016/j.neucom.2024.128315604:COnline publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128315
Zhang CLi WMo YTang WLi HZeng Z(2024)BHRAM: a knowledge graph embedding model based on bidirectional and heterogeneous relational attention mechanismApplied Intelligence10.1007/s10489-024-06212-455:4Online publication date: 30-Dec-2024
https://dl.acm.org/doi/10.1007/s10489-024-06212-4
Liang QWang WYu JBao F(2024)Effective Knowledge Graph Embedding with Quaternion Convolutional NetworksNatural Language Processing and Chinese Computing10.1007/978-981-97-9437-9_15(183-196)Online publication date: 2-Nov-2024
https://dl.acm.org/doi/10.1007/978-981-97-9437-9_15
Liu XLi YWang PMei Q(2024)Construction of a Large-Scale Maritime Elements Semantic Schema Based on Heterogeneous Graph ModelsSpatial Data and Intelligence10.1007/978-981-97-2966-1_11(132-151)Online publication date: 25-Apr-2024
https://dl.acm.org/doi/10.1007/978-981-97-2966-1_11

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents