research-article

Multi-Concept Representation Learning for Knowledge Graph Completion

Authors:

Baocai YinAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 17, Issue 1

Article No.: 11, Pages 1 - 19

https://doi.org/10.1145/3533017

Published: 20 February 2023 Publication History

Abstract

Knowledge Graph Completion (KGC) aims at inferring missing entities or relations by embedding them in a low-dimensional space. However, most existing KGC methods generally fail to handle the complex concepts hidden in triplets, so the learned embeddings of entities or relations may deviate from the true situation. In this article, we propose a novel Multi-concept Representation Learning (McRL) method for the KGC task, which mainly consists of a multi-concept representation module, a deep residual attention module, and an interaction embedding module. Specifically, instead of the single-feature representation, the multi-concept representation module projects each entity or relation to multiple vectors to capture the complex conceptual information hidden in them. The deep residual attention module simultaneously explores the inter- and intra-connection between entities and relations to enhance the entity and relation embeddings corresponding to the current contextual situation. Moreover, the interaction embedding module further weakens the noise and ambiguity to obtain the optimal and robust embeddings. We conduct the link prediction experiment to evaluate the proposed method on several standard datasets, and experimental results show that the proposed method outperforms existing state-of-the-art KGC methods.

References

[1]

Luyi Bai, Xiangnan Ma, Mingcheng Zhang, and Wenting Yu. 2021. TPmod: A tendency-guided prediction model for temporal knowledge graph completion. ACM Transactions on Knowledge Discovery from Data 15, 3 (2021), 1–17.

Digital Library

[2]

Ivana Balažević, Carl Allen, and Timothy Hospedales. 2019a. Hypernetwork knowledge graph embeddings. In Proceedings of the International Conference on Artificial Neural Networks. 553–565.

[3]

Ivana Balažević, Carl Allen, and Timothy M. Hospedales. 2019b. Tucker: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 5185–5194.

[4]

Trapit Bansal, Da Cheng Juan, Sujith Ravi, and Andrew Mccallum. 2019. A2N: Attending to neighbors for knowledge graph inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1777–1786.

[5]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. 1247–1250.

Digital Library

[6]

Antoine Bordes, Sumit Chopra, and Jason Weston. 2014. Question answering with subgraph embeddings. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 615–620.

[7]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems. 2787–2795.

[8]

Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, and Qingming Huang. 2021. Dual quaternion knowledge graph embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence. 6894–6902.

[9]

Z. Cao, Q. Xu, Z. Yang, X. Cao, and Q. Huang. 2022. Geometry interaction knowledge graph embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence.

[10]

Ines Chami, Adva Wolf, Da Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher Ré. 2020. Low-dimensional hyperbolic knowledge graph embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6901–6914.

[11]

Feihu Che, Dawei Zhang, Jianhua Tao, Mingyue Niu, and Bocheng Zhao. 2020. Parame: Regarding neural network parameters as relation embeddings for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 2774–2781.

[12]

Jianpeng Cheng, Zhongyuan Wang, Ji-Rong Wen, Jun Yan, and Zheng Chen. 2015. Contextual text understanding in distributional semantic space. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 133–142.

Digital Library

[13]

Jeffrey Dalton, Laura Dietz, and James Allan. 2014. Entity query feature expansion using knowledge base links. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. 365–374.

Digital Library

[14]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence. 1811–1818.

[15]

Takuma Ebisu and Ryutaro Ichise. 2019. Graph pattern entity ranking model for knowledge graph completion. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 988–997.

[16]

Takuma Ebisu and Ryutaro Ichise. 2020. Generalized translation-based embedding of knowledge graph. IEEE Transactions on Knowledge and Data Engineering 32, 5 (2020), 941–951.

[17]

Robert I. Goldblatt. 1974. Semantic analysis of orthologic. Journal of Philosophical Logic 3, 1–2 (1974), 19–35.

[18]

Niannian Guan, Dandan Song, and Lejian Liao. 2019. Knowledge graph embedding with concepts. Knowledge-Based Systems 164 (2019), 38–44.

[19]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning. 448–456.

[20]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. 1–13.

[21]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 2181–2187.

[22]

Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the International Conference on Machine Learning. 2168–2178.

[23]

Yi Ma, Paul A. Crook, Ruhi Sarikaya, and Eric Fosler-Lussier. 2015. Knowledge graph inference for spoken dialog systems. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing. 5346–5350.

[24]

Farzaneh Mahdisoltani, Joanna Biega, and Fabian Suchanek. 2015. YAGO3: A knowledge base from multilingual wikipedias. In Proceedings of the Biennial Conference on Innovative Data Systems Research.

[25]

George A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11 (1995), 39–41.

Digital Library

[26]

Tom Mitchell, William Cohen, Estevam Hruschka, Partha Talukdar, Bishan Yang, Justin Betteridge, Andrew Carlson, Bhavana Dalvi, Matt Gardner, Bryan Kisiel, and J. Krishnamurthy. 2018. Never-ending learning. Communications of the ACM 61, 5 (2018), 103–115.

Digital Library

[27]

Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2018. A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 327–333.

[28]

M. Nickel, L. Rosasco, and T. Poggio. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence. 1955–1961.

[29]

Maximilian Nickel, Volker Tresp, and Hans Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the International Conference on Machine Learning. 809–816.

[30]

Jongchan Park, Sanghyun Woo, Joon-Young Lee, and In So Kweon. 2018. BAM: Bottleneck attention module. In Proceedings of the British Machine Vision Conference. 1–14.

[31]

Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo. 2021. Knowledge graph embedding for link prediction: A comparative analysis. ACM Transactions on Knowledge Discovery from Data 15, 2 (2021), 1–49.

Digital Library

[32]

Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In Proceedings of the European Semantic Web Conference. Springer, 593–607.

Digital Library

[33]

Shaozhi, Dai, YanchunLiang, ShuyanLiu, YingWang, WenleShao, Xixun Lin, and Xiaoyue Feng. 2018. Learning entity and relation embeddings with entity description for knowledge graph completion. In Proceedings of the 2018 2nd International Conference on Artificial Intelligence: Technologies and Applications. 202–205.

[34]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929–1958.

Digital Library

[35]

Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. YAGO: A core of semantic knowledge. In Proceedings of the International Conference on World Wide Web. 697–706.

Digital Library

[36]

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the International Conference on Learning Representations. 1–18.

[37]

Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. 57–66.

[38]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the International Conference on Machine Learning. 2071–2080.

[39]

Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, Nilesh Agrawal, and Partha Talukdar. 2020. Interacte: Improving convolution-based knowledge graph embeddings by increasing feature interactions. In Proceedings of the AAAI Conference on Artificial Intelligence. 3009–3016.

[40]

Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29, 12 (2017), 2724–2743.

[41]

Shen Wang, Xiaokai Wei, Cicero Nogueira Dos Santos, Zhiguo Wang, Ramesh M. Nallapati, Andrew Arnold, Bing Xiang, Philip S. Yu, and Isabel F. Cruz. 2021. Mixed-curvature multi-relational graph neural network for knowledge graph completion. In Proceedings of the Web Conference. 1761–1771.

Digital Library

[42]

Yashen Wang and Huanhuan Zhang. 2021. HARP: A novel hierarchical attention model for relation prediction. ACM Transactions on Knowledge Discovery from Data 15, 2 (2021), 1–22.

Digital Library

[43]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI Conference on Artificial Intelligence. 1112–1119.

[44]

Chenfei Wu, Jinlai Liu, Xiaojie Wang, and Ruifan Li. 2019. Differential networks for visual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence. 8997–9004.

Digital Library

[45]

Han Xiao, Minlie Huang, Lian Meng, and Xiaoyan Zhu. 2017. SSP: Semantic space projection for knowledge graph embedding with text descriptions. In Proceedings of the AAAI Conference on Artificial Intelligence. 3104–3110.

[46]

Z. Xie, G. Zhou, J. Liu, and X. Huang. 2020. ReInceptionE: Relation-aware inception network with joint local-global structural information for knowledge graph embedding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5929–5939.

[47]

Jiacheng Xu, Kan Chen, Xipeng Qiu, and Xuanjing Huang. 2017. Knowledge graph representation with jointly structural and textual encoding. In Proceedings of the International Joint Conference on Artificial Intelligence. 1318–1324.

[48]

Bishan Yang, Wen Tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the International Conference on Learning Representations. 1–13.

[49]

Zhou Yu, Jun Yu, Jianping Fan, and Dacheng Tao. 2017. Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In Proceedings of the International Conference on Computer Vision. 1821–1830.

[50]

Wen Zhang, Bibek Paudel, Wei Zhang, Abraham Bernstein, and Huajun Chen. 2019. Interaction embeddings for prediction and explanation in knowledge graphs. In Proceedings of the ACM International Conference on Web Search and Data Mining. 96–104.

Digital Library

[51]

Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. 3065–3072.

[52]

Zhao Zhang, Fuzhen Zhuang, Hengshu Zhu, Zhiping Shi, Hui Xiong, and Qing He. 2020. Relational graph neural network with hierarchical attention for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 9612–9619.

Cited By

Yan XYi YShi WTian HSu X(2024)Improvement of Web Semantic and Transformer-Based Knowledge Graph Completion in Low-Dimensional SpacesInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.33691920:1(1-18)Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.4018/IJSWIS.336919
Feng PZhang XWu HWang YYang ZOuyang D(2024)Link Prediction Based on Feature Mapping and Bi-Directional ConvolutionApplied Sciences10.3390/app1405208914:5(2089)Online publication date: 2-Mar-2024
https://doi.org/10.3390/app14052089
Mao QLi ZLiu QWu LZhang HChen E(2024)Promoting Machine Abilities of Discovering and Utilizing Knowledge in a Unified Zero-Shot Learning ParadigmACM Transactions on Knowledge Discovery from Data10.1145/370044419:1(1-26)Online publication date: 30-Nov-2024
https://doi.org/10.1145/3700444
Show More Cited By

Index Terms

Multi-Concept Representation Learning for Knowledge Graph Completion
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Separate-and-Aggregate: A Transformer-Based Patch Refinement Model for Knowledge Graph Completion
Advanced Data Mining and Applications
Abstract
Knowledge graph completion (KGC) is the task of inferencing missing facts from any given knowledge graphs (KG). Previous KGC methods typically represent knowledge graph entities and relations as trainable continuous embeddings and fuse the ...
Learning Entity Type Embeddings for Knowledge Graph Completion
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Missing data is a severe problem for algorithms that operate over knowledge graphs (KGs). Most previous research in KG completion has focused on the problem of inferring missing entities and missing relation types between entities. However, in addition ...
Graph attention network with dynamic representation of relations for knowledge graph completion
Abstract
Knowledge graph completion (KGC) aims to predict the missing element in a triple based on known triples or facts. Recently, plenty of representation learning methods for KGC have achieved the promising performance, especially ones ...
Highlights
- Graph Attention Network with Dynamic Representation of Relations (DRR-GAT) is proposed.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 17, Issue 1

January 2023

375 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3572846

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 February 2023

Online AM: 30 April 2022

Accepted: 20 April 2022

Revised: 31 January 2022

Received: 17 November 2021

Published in TKDD Volume 17, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key R&D Program of China
National Natural Science Foundation of China
Beijing Municipal Science and Technology

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
1,309
Total Downloads

Downloads (Last 12 months)541
Downloads (Last 6 weeks)49

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yan XYi YShi WTian HSu X(2024)Improvement of Web Semantic and Transformer-Based Knowledge Graph Completion in Low-Dimensional SpacesInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.33691920:1(1-18)Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.4018/IJSWIS.336919
Feng PZhang XWu HWang YYang ZOuyang D(2024)Link Prediction Based on Feature Mapping and Bi-Directional ConvolutionApplied Sciences10.3390/app1405208914:5(2089)Online publication date: 2-Mar-2024
https://doi.org/10.3390/app14052089
Mao QLi ZLiu QWu LZhang HChen E(2024)Promoting Machine Abilities of Discovering and Utilizing Knowledge in a Unified Zero-Shot Learning ParadigmACM Transactions on Knowledge Discovery from Data10.1145/370044419:1(1-26)Online publication date: 30-Nov-2024
https://doi.org/10.1145/3700444
Yang GZhang LLiu YXie HMao Z(2024)Exploiting Pre-Trained Language Models for Black-Box Attack against Knowledge Graph EmbeddingsACM Transactions on Knowledge Discovery from Data10.1145/368885019:1(1-14)Online publication date: 29-Nov-2024
https://doi.org/10.1145/3688850
Sun KJiang HHu YYin B(2024)Incorporating Multi-Level Sampling with Adaptive Aggregation for Inductive Knowledge Graph CompletionACM Transactions on Knowledge Discovery from Data10.1145/364482218:5(1-16)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3644822
Rong HQian MMa TJin DSheng V(2024)CoBjeason: Reasoning Covered Object in Image by Multi-Agent Collaboration Based on Informed Knowledge GraphACM Transactions on Knowledge Discovery from Data10.1145/364356518:5(1-56)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3643565
Chen CLee CHuang SPeng W(2024)Credit Card Fraud Detection via Intelligent Sampling and Self-supervised LearningACM Transactions on Intelligent Systems and Technology10.1145/364128315:2(1-29)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3641283
Ye ZXie XAi QLiu YWang ZSu WZhang M(2024)Relevance Feedback with Brain SignalsACM Transactions on Information Systems10.1145/363787442:4(1-37)Online publication date: 9-Feb-2024
https://dl.acm.org/doi/10.1145/3637874
Liu JChen CLee CHuang S(2024)Evolving Knowledge Graph Representation Learning with Multiple Attention Strategies for Citation Recommendation SystemACM Transactions on Intelligent Systems and Technology10.1145/363527315:2(1-26)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3635273
Wang JWang BGao JLi XHu YYin B(2024)QDN: A Quadruplet Distributor Network for Temporal Knowledge Graph CompletionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.327423035:10(14018-14030)Online publication date: Oct-2024
https://doi.org/10.1109/TNNLS.2023.3274230
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents