MAKE:Knowledge Graph Embedding via Multi-Attention neural network
Pages 347 - 352
Abstract
Knowledge graph embedding is a popular method to solve the incompleteness of knowledge graph. At present, research on knowledge graph embedding based on neural network has achieved remarkable results, but most models ignore the influence of the correlation among with subject entity and relation and object entity within triple. Existing attention-based models take into account the effect of correlation, but perform moderately. In this paper, we propose a multi-attention neural network-based embedding model, named MAKE, which utilizes a novel multi-attention mechanism to generate feature maps of triples by computing correlations within triple. To fully exploit the performance of the multi-attention mechanism, MAKE uses a trainable batch normalization method and a novel composite loss function to improve the model learning ability. Evaluation results on FB15K-237 and WN18RR standard datasets show that our MAKE achieves better performance than previous state-of-the-art knowledge graph embedding models.
References
[1]
Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li. 2020. Realistic re-evaluation of knowledge graph completion methods: An experimental study. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 1995–2010.
[2]
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data. 1247–1250.
[3]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems 26 (2013).
[4]
Feihu Che, Dawei Zhang, Jianhua Tao, Mingyue Niu, and Bocheng Zhao. 2020. Parame: Regarding neural network parameters as relation embeddings for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 2774–2781.
[5]
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[6]
Larry Heck and Hongzhao Huang. 2014. Deep learning of knowledge graph embeddings for semantic parsing of twitter dialogs. In 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, 597–601.
[7]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. PMLR, 448–456.
[8]
Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long papers). 687–696.
[9]
Dan Jiang, Ronggui Wang, Juan Yang, and Lixia Xue. 2021. Kernel multi-attention neural network for knowledge graph embedding. Knowledge-Based Systems 227 (2021), 107188.
[10]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).
[11]
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence.
[12]
Farzaneh Mahdisoltani, Joanna Biega, and Fabian Suchanek. 2014. Yago3: A knowledge base from multilingual wikipedias. In 7th biennial conference on innovative data systems research. CIDR Conference.
[13]
George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (1995), 39–41.
[14]
Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2017. A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121(2017).
[15]
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Icml.
[16]
Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. 2020. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In Proceedings of the 58th annual meeting of the association for computational linguistics. 4498–4507.
[17]
Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. Advances in neural information processing systems 26 (2013).
[18]
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197(2019).
[19]
Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality. 57–66.
[20]
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In International conference on machine learning. PMLR, 2071–2080.
[21]
Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung, 2019. A capsule network-based embedding model for knowledge graph completion and search personalization. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2180–2189.
[22]
Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and Dekang Lin. 2014. Knowledge base completion via search-based question answering. In Proceedings of the 23rd international conference on World wide web. 515–526.
[23]
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2014. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575(2014).
[24]
Ganggao Zhu and Carlos A Iglesias. 2018. Exploiting semantic similarity for named entity disambiguation in knowledge graphs. Expert Systems with Applications 101 (2018), 8–24.
Index Terms
- MAKE:Knowledge Graph Embedding via Multi-Attention neural network
Recommendations
Mixed-Curvature Multi-Relational Graph Neural Network for Knowledge Graph Completion
WWW '21: Proceedings of the Web Conference 2021Knowledge graphs (KGs) have gradually become valuable assets for many AI applications. In a KG, a node denotes an entity, and an edge (or link) denotes a relationship between the entities represented by the nodes. Knowledge graph completion infers and ...
Multisource hierarchical neural network for knowledge graph embedding
AbstractLink prediction for knowledge graphs aims to obtain missing nodes in triples. In recent years, link prediction methods have made specific achievements in knowledge graph embedding. However, knowledge graphs are characteristic of the heterogeneity ...
Kernel multi-attention neural network for knowledge graph embedding
AbstractLink prediction is the problem of predicting missing link between entities and relations for knowledge graph. In recent years, some tasks have achieved great success for link prediction, but these tasks are far from expanding entity ...
Comments
Information & Contributors
Information
Published In
December 2022
406 pages
ISBN:9781450399067
DOI:10.1145/3578741
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 March 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Science and Technology Commission of Shanghai Municipality
- Shandong Provincial Natural Science Foundation
Conference
MLNLP 2022
MLNLP 2022: 2022 5th International Conference on Machine Learning and Natural Language Processing
December 23 - 25, 2022
Sanya, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 41Total Downloads
- Downloads (Last 12 months)13
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format