research-article

Knowledge Graph Embedding: A Locally and Temporally Adaptive Translation-Based Approach

Authors:

Xueqi ChengAuthors Info & Claims

ACM Transactions on the Web (TWEB), Volume 12, Issue 2

Article No.: 8, Pages 1 - 33

https://doi.org/10.1145/3132733

Published: 22 December 2017 Publication History

Abstract

A knowledge graph is a graph with entities of different types as nodes and various relations among them as edges. The construction of knowledge graphs in the past decades facilitates many applications, such as link prediction, web search analysis, question answering, and so on. Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, for example, TransE, TransH, and TransR, learn the embedding representation by defining a global margin-based loss function over the data. However, the loss function is determined during experiments whose parameters are examined among a closed set of candidates. Moreover, embeddings over two knowledge graphs with different entities and relations share the same set of candidates, ignoring the locality of both graphs. This leads to the limited performance of embedding related applications. In this article, a locally adaptive translation method for knowledge graph embedding, called TransA, is proposed to find the loss function by adaptively determining its margin over different knowledge graphs. Then the convergence of TransA is verified from the aspect of its uniform stability. To make the embedding methods up-to-date when new vertices and edges are added into the knowledge graph, the incremental algorithm for TransA, called iTransA, is proposed by adaptively adjusting the optimal margin over time. Experiments on four benchmark data sets demonstrate the superiority of the proposed method, as compared to the state-of-the-art ones.

References

[1]

Yoshua Bengio, Aaron Courville, and Pierre Vincent. 2013. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 8 (2013), 1798--1828.

Digital Library

[2]

James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, Feb (2012), 281--305.

Digital Library

[3]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 1247--1250.

Digital Library

[4]

Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 127--135.

[5]

Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2014. A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94, 2 (2014), 233--259.

Digital Library

[6]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems. 2787--2795.

Digital Library

[7]

Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Proceedings of the Conference on Artificial Intelligence.

Digital Library

[8]

Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. 1992. A training algorithm for optimal margin classifiers. In Proceedings of the 5th Annual Workshop on Computational Learning Theory. ACM, 144--152.

Digital Library

[9]

Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. J. Mach. Learn. Res. 2 (2002), 499--526.

Digital Library

[10]

Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Toward an architecture for never-ending language learning. In Proceedings of AAAI Conference on Artificial Intelligence.

Digital Library

[11]

Kai-Wei Chang, Wen-tau Yih, and Christopher Meek. 2013. Multi-relational latent semantic analysis. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP’13). 1602--1612.

[12]

Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. 2014. Typed tensor decomposition of knowledge bases for relation extraction. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1568--1579.

[13]

Olivier Chapelle, Vladimir Vapnik, Olivier Bousquet, and Sayan Mukherjee. 2002. Choosing multiple parameters for support vector machines. Mach. Learn. 46, 1 (2002), 131--159.

Digital Library

[14]

Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. 2000. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 4 (2000), 1253--1278.

Digital Library

[15]

Foo Chuan-sheng Do, B. Chuong, and Andrew Y. Ng. 2007. Efficient multiple hyperparameter learning for log-linear models. In Advances in Neural Information Processing Systems. 377--384.

Digital Library

[16]

Huyen Do, Alexandros Kalousis, and Melanie Hilario. 2009. Feature Weighting Using Margin and Radius Based Error Bound Optimization in SVMs. Springer.

[17]

Huyen Do, Alexandros Kalousis, Jun Wang, and Adam Woznica. 2012. A metric learning perspective of SVM: On the relation of LMNN and SVM. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 308--317.

[18]

Sergio Duarte Torres, Ingmar Weber, and Djoerd Hiemstra. 2014. Analysis of search and browsing behavior of young users on the web. ACM Trans. Web (TWEB) 8, 2 (2014), 7.

Digital Library

[19]

Daniel M. Dunlavy, Tamara G. Kolda, and Evrim Acar. 2011. Temporal link prediction using matrix and tensor factorizations. ACM Trans. Knowl. Discov. Data (TKDD) 5, 2 (2011), 10.

Digital Library

[20]

Miao Fan, Qiang Zhou, Emily Chang, and Thomas Fang Zheng. 2014. Transition-based knowledge graph embedding with relational mapping properties. In Proceedings of the 28th Pacific Asia Conference on Language, Information, and Computation. 328--337.

[21]

Miao Fan, Qiang Zhou, Thomas Fang Zheng, and Ralph Grishman. 2015. Large margin nearest neighbor embedding for knowledge representation. arXiv:1504.01684 (2015).

[22]

Thomas Franz, Antje Schultz, Sergej Sizov, and Steffen Staab. 2009. Triplerank: Ranking Semantic Web Data by Tensor Decomposition. Springer.

[23]

Kun Gai, Guangyun Chen, and Chang-shui Zhang. 2010. Learning kernels with radiuses of minimum enclosing balls. In Advances in Neural Information Processing Systems. 649--657.

Digital Library

[24]

Hong Huang, Jie Tang, Lu Liu, JarDer Luo, and Xiaoming Fu. 2015. Triadic closure pattern analysis and prediction in social networks. IEEE Trans. Knowl. Data Eng. 27, 12 (2015), 3374--3389.

Digital Library

[25]

Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, and Guillaume R. Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems. 3167--3175.

Digital Library

[26]

Yantao Jia, Yuanzhuo Wang, Xueqi Cheng, Xiaolong Jin, and Jiafeng Guo. 2014. OpenKN: An open knowledge computational engine for network big data. In Proceedings of the 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’14). IEEE, 657--664.

[27]

Yantao Jia, Yuanzhuo Wang, Xiaolong Jin, Zeya Zhao, and Xueqi Cheng. 2017. Link inference in dynamic heterogeneous information network: A knapsack-based approach. IEEE Trans. Comput. Soc. Syst. (2017).

[28]

Yantao Jia, Yuanzhuo Wang, Hailun Lin, Xiaolong Jin, and Xueqi Cheng. Locally adaptive translation for knowledge graph embedding. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16), 992–998.

Digital Library

[29]

Yan-Tao Jia, Yuan-Zhuo Wang, and Xue-Qi Cheng. 2015. Learning to predict links by integrating structure and interaction information in microblogs. J. Comput. Sci. Technol. 30, 4 (2015), 829--842.

[30]

Henk A. L. Kiers. 2000. Towards a standardized notation and terminology in multiway analysis. J. Chemometr. 14, 3 (2000), 105--122.

[31]

Ravi Kumar, Jasmine Novak, and Andrew Tomkins. 2010. Structure and evolution of online social networks. In Link Mining: Models, Algorithms, and Applications. Springer, 337--357.

[32]

Jure Leskovec, Lars Backstrom, Ravi Kumar, and Andrew Tomkins. 2008. Microscopic evolution of social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 462--470.

Digital Library

[33]

Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2015. Modeling relation paths for representation learning of knowledge bases. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 705--714.

[34]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2181--2187.

Digital Library

[35]

Dawei Liu, Yuanzhuo Wang, Yantao Jia, Jingyuan Li, and Zhihua Yu. 2014. LSDH: A hashing approach for large-scale link prediction in microblogs. In Proceedings of the 28th AAAI Conference on Artificial Intelligence.

Digital Library

[36]

Dougal Maclaurin, David Duvenaud, and Ryan P. Adams. 2015. Gradient-based hyperparameter optimization through reversible learning. In Proceedings of the 32nd International Conference on Machine Learning.

Digital Library

[37]

George A. Miller. 1995. WordNet: A lexical database for English. Commun. ACM 38, 11 (1995), 39--41.

Digital Library

[38]

T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohamed, N. Nakashole, E. Platanios, A. Ritter, M. Samadi, B. Settles, R. Wang, D. Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, and J. Welling. 2015. Never-ending learning. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15). 1859--1865.

Digital Library

[39]

Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. 2016. Neighborhood mixture model for knowledge base completion. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. 40--50.

[40]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2012. Factorizing yago: Scalable machine learning for linked data. In Proceedings of the 21st International Conference on World Wide Web. ACM, 271--280.

Digital Library

[41]

Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro, and Karthik Sridharan. 2010. Learnability, stability and uniform convergence. J. Mach. Learn. Res. 11 (2010), 2635--2670.

Digital Library

[42]

Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems. 926--934.

Digital Library

[43]

Ao-Jan Su, Y. Charlie Hu, Aleksandar Kuzmanovic, and Cheng-Kok Koh. 2014. How to improve your search engine ranking: Myths and reality. ACM Trans. Web (TWEB) 8, 2 (2014), 8.

Digital Library

[44]

Ilya Sutskever, Joshua B. Tenenbaum, and Ruslan R. Salakhutdinov. 2009. Modelling relational data using Bayesian clustered tensor factorization. In Advances in Neural Information Processing Systems. 1821--1828.

Digital Library

[45]

Jie Tang, Tiancheng Lou, Jon Kleinberg, and Sen Wu. 2016. Transfer learning to infer social ties across heterogeneous networks. ACM Trans. Info. Syst. (TOIS) 34, 2 (2016), 7.

Digital Library

[46]

Vladimir Vapnik. 2013. The Nature of Statistical Learning Theory. Springer Science 8 Business Media.

Digital Library

[47]

Chenguang Wang, Yangqiu Song, Ahmed El-Kishky, Dan Roth, Ming Zhang, and Jiawei Han. 2015. Incorporating world knowledge to document clustering via heterogeneous information networks. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1215--1224.

Digital Library

[48]

Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han. 2016. Text classification with heterogeneous information network kernels. In Proceedings of the 30th AAAI Conference on Artificial Intelligence.

Digital Library

[49]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Citeseer, 1112--1119.

Digital Library

[50]

Kilian Q. Weinberger and Lawrence K. Saul. 2008. Fast solvers and efficient implementations for distance metric learning. In Proceedings of the 25th International Conference on Machine Learning. ACM, 1160--1167.

Digital Library

[51]

Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10 (2009), 207--244.

Digital Library

[52]

Wentao Wu, Hongsong Li, Haixun Wang, and Kenny Q Zhu. 2012. Probase: A probabilistic taxonomy for text understanding. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. ACM, 481--492.

Digital Library

[53]

Denghui Zhang, Manling Li, Yantao Jia, Yuanzhuo Wang, and Xueqi Cheng. 2017. Efficient parallel translating embedding for knowledge graphs. In Proceedings of the 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI’17). IEEE, 460--468.

Digital Library

[54]

Jing Zhang, Zhanpeng Fang, Wei Chen, and Jie Tang. 2015. Diffusion of following links in microblogging networks. IEEE Trans. Knowl. Data Eng. 27, 8 (2015), 2093--2106.

Digital Library

[55]

Zeya Zhao, Yantao Jia, and Yuanzhuo Wang. 2014. Content-structural relation inference in knowledge base. In Proceedings of the 28th AAAI Conference on Artificial Intelligence.

Digital Library

Cited By

Abadeh M(2024)Knowledge-enhanced software refinement: leveraging reinforcement learning for search-based quality engineeringAutomated Software Engineering10.1007/s10515-024-00456-731:2Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10515-024-00456-7
Zhang CCao X(2023)Biological gene extraction path based on knowledge graph and natural language processingFrontiers in Genetics10.3389/fgene.2022.108637913Online publication date: 13-Jan-2023
https://doi.org/10.3389/fgene.2022.1086379
Susanti FMaulidevi NSurendro K(2023)Improving the Efficiency of Link Prediction on Handling Incomplete Knowledge Graph Using ClusteringProceedings of the 2023 12th International Conference on Software and Computer Applications10.1145/3587828.3587830(8-13)Online publication date: 23-Feb-2023
https://dl.acm.org/doi/10.1145/3587828.3587830
Show More Cited By

Index Terms

Knowledge Graph Embedding: A Locally and Temporally Adaptive Translation-Based Approach
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Reasoning about belief and knowledge
      2. Temporal reasoning
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Random projections and metric embeddings

Recommendations

Locally adaptive translation for knowledge graph embedding
AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, e.g., TransE and TransH, learn embedding representation by defining a global margin-based ...
FedE: Embedding Knowledge Graphs in Federated Setting
IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge Graphs

Knowledge graphs (KGs) become widespread and many organizations construct as well as maintain their own knowledge graphs. Same as the data isolation which has been a long-standing problem, knowledge graph isolation is common in real knowledge graph ...
Knowledge Graph Embedding with Diversity of Structures
WWW '17 Companion: Proceedings of the 26th International Conference on World Wide Web Companion

In recent years, different web knowledge graphs, both free and commercial, have been created. Knowledge graphs use relations between entities to describe facts in the world. We engage in embedding a large scale knowledge graph into a continuous vector ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on the Web

ACM Transactions on the Web Volume 12, Issue 2

May 2018

174 pages

ISSN:1559-1131

EISSN:1559-114X

DOI:10.1145/3176641

Editor:
Brian D. Davison
Lehigh University, USA

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2017

Accepted: 01 August 2017

Revised: 01 August 2017

Received: 01 July 2016

Published in TWEB Volume 12, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Grand Fundamental Research 973 Program of China
National Key Research and Development Program of China
National High-tech R8D Program (863 Program)
National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
1,182
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)3

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Abadeh M(2024)Knowledge-enhanced software refinement: leveraging reinforcement learning for search-based quality engineeringAutomated Software Engineering10.1007/s10515-024-00456-731:2Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10515-024-00456-7
Zhang CCao X(2023)Biological gene extraction path based on knowledge graph and natural language processingFrontiers in Genetics10.3389/fgene.2022.108637913Online publication date: 13-Jan-2023
https://doi.org/10.3389/fgene.2022.1086379
Susanti FMaulidevi NSurendro K(2023)Improving the Efficiency of Link Prediction on Handling Incomplete Knowledge Graph Using ClusteringProceedings of the 2023 12th International Conference on Software and Computer Applications10.1145/3587828.3587830(8-13)Online publication date: 23-Feb-2023
https://dl.acm.org/doi/10.1145/3587828.3587830
Wei YChen WWen SLiu AZhao L(2023)Knowledge graph incremental embedding for unseen modalitiesKnowledge and Information Systems10.1007/s10115-023-01868-965:9(3611-3631)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1007/s10115-023-01868-9
Zhang CLu K(2022)Knowledge Graph Completion Algorithm Based on Probabilistic Fuzzy Information Aggregation and Natural Language Processing TechnologyMathematics10.3390/math1023457810:23(4578)Online publication date: 2-Dec-2022
https://doi.org/10.3390/math10234578
Wu TKhan AYong MQi GWang M(2022)Efficiently embedding dynamic knowledge graphsKnowledge-Based Systems10.1016/j.knosys.2022.109124250(109124)Online publication date: Aug-2022
https://doi.org/10.1016/j.knosys.2022.109124
Saebi MKreig SZhang CJiang MKajdanowicz TChawla N(2022)Heterogeneous relational reasoning in knowledge graphs with reinforcement learningInformation Fusion10.1016/j.inffus.2022.07.00188(12-21)Online publication date: Dec-2022
https://doi.org/10.1016/j.inffus.2022.07.001
Wang MQiu LWang X(2021)A Survey on Knowledge Graph Embeddings for Link PredictionSymmetry10.3390/sym1303048513:3(485)Online publication date: 16-Mar-2021
https://doi.org/10.3390/sym13030485
Xu YSun SZhang HYi CMiao YYang DMeng XHu YWang KMin HSong HMiao C(2021)Time-Aware Graph Embedding: A Temporal Smoothness and Task-Oriented ApproachACM Transactions on Knowledge Discovery from Data10.1145/348024316:3(1-23)Online publication date: 22-Oct-2021
https://dl.acm.org/doi/10.1145/3480243
Chen YLiu JZhang ZWen SXiong W(2021)MöbiusE: Knowledge Graph Embedding on Möbius ringKnowledge-Based Systems10.1016/j.knosys.2021.107181(107181)Online publication date: Jun-2021
https://doi.org/10.1016/j.knosys.2021.107181
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents