research-article

TPmod: A Tendency-Guided Prediction Model for Temporal Knowledge Graph Completion

Authors:

Mingcheng Zhang,

Wenting YuAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 15, Issue 3

Article No.: 41, Pages 1 - 17

https://doi.org/10.1145/3443687

Published: 21 April 2021 Publication History

Abstract

Temporal knowledge graphs (TKGs) have become useful resources for numerous Artificial Intelligence applications, but they are far from completeness. Inferring missing events in temporal knowledge graphs is a fundamental and challenging task. However, most existing methods solely focus on entity features or consider the entities and relations in a disjoint manner. They do not integrate the features of entities and relations in their modeling process. In this paper, we propose TPmod, a tendency-guided prediction model, to predict the missing events for TKGs (extrapolation). Differing from existing works, we propose two definitions for TKGs: the Goodness of relations and the Closeness of entity pairs. More importantly, inspired by the attention mechanism, we propose a novel tendency strategy to guide our aggregated process. It integrates the features of entities and relations, and assigns varying weights to different past events. What is more, we select the Gate Recurrent Unit (GRU) as our sequential encoder to model the temporal dependency in TKGs. Besides, the Softmax function is employed to generate the final decreasing group of candidate entities. We evaluate our model on two TKG datasets: GDELT-5 and ICEWS-250. Experimental results show that our method has a significant and consistent improvement compared to state-of-the-art baselines.

References

[1]

Giang Hoang Nguyen, Jhon Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-time dynamic network embeddings. In Companion Proceedings of the The Web Conference 2018. 969–976.

Digital Library

[2]

Caiming Xiong, Stephen Merity, and Richard Scoher. 2016. Dynamic memory networks for visual and textual question answering. In Proceedings of the International Conference on Machine Learning. 2397–2406.

[3]

Caiming Xiong, Victor Zhong, and Richaid Socher. 2017. Dynamic coattention networks for question answering. In Proceedings of the International Conference on Learning Representations.

[4]

Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. 2019. DyRep: Learning representations over dynamic graphs. In Proceedings of the International Conference on Learning Representations.

[5]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 1247–1250.

Digital Library

[6]

George A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11 (1995), 39–41.

Digital Library

[7]

Zhiqing Sun, Zhihong Deng, Jianyun Nie, and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the International Conference on Learning Representations.

[8]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on Machine Learning. 2071–2080.

[9]

Seyed Mehran Kazemi and David Poole. 2018. SimplE embedding for link prediction in knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 4289–4300.

[10]

Kalev Leetaru and Philip A. Schrodt. 2013. GDELT: Global data on events, location, and tone, 1979–2012. In Proceedings of the International Studies Association Annual Convention.

[11]

Elizabeth Boschee, Janin Lautenschläger, Sean O'Brien, Stephen M. Shellman, James Starz, and Michael D. Ward. 2015. ICEWS Coded Event Data. Harvard Dataverse 12. Retrieved from https://doi.org/10.7910/DVN/28075.

[12]

Philip A. Schrodt. 2012. CAMEO: Conflict and Mediation Event Observations Event and Actor Codebook. Pennsylvania State University, University Park, PA.

[13]

Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. 2017. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In Proceedings of the 34th Proceedings of the International Conference on Machine Learning. Vol. 70. ACM, 3462–3471.

[14]

Woojeong Jin, Changlin Zhang, Pedro Szekely, and Xiang Ren. 2019. Recurrent event network for reasoning over temporal knowledge graphs. In Proceedings of the 7th International Conference on Learning Representations.

[15]

Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29, 12 (2017), 2724–2743.

[16]

Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S. Yu. 2020. A survey on knowledge graphs: Representation, acquisition and applications. arXiv:2002.00388. Retrieved from https://arxiv.org/abs/2002.00388.

[17]

Bianchi Federico, Gaetano Rossiello, Luca Costabello, Matteo Palmonari, and Pasquale Minervini. 2020. Knowledge graph embeddings and explainable AI. arXiv:2004.14843. Retrieved from https://arxiv.org/abs/2004.14843.

[18]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems. Vol. 2. ACM, 2787–2795.

[19]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 1112–1119.

[20]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 2181–2187.

[21]

Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016. Representation learning of knowledge graphs with hierarchical types. In Proceedings of the International Joint Conference on Artificial Intelligence. 2965–2971.

[22]

Ruobing Xie, Zhiyuan Liu, Jia Jia, Hanbo Luan, and Maosong Sun. 2016. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 189–205.

[23]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th Proceedings of the International Conference on Machine Learning. 809–816.

[24]

Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the International Conference on Learning Representations.

[25]

Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 1955–1961.

[26]

Ivana Balažević, Carl Allen, and Timothy M. Hospedales. 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the International Joint Conference on Natural Language Processing.

[27]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence.

[28]

Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. Hypernetwork knowledge graph embeddings. In Proceedings of the International Conference on Artificial Neural Networks. I. Tetko, V. Kůrková, P. Karpov, F. Theis (Eds.), Lecture Notes in Computer Science, Vol. 11731, Springer, Cham, 553–565.

[29]

Julien Leblay and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In Companion Proceedings of the The Web Conference 2018. 1771–1776.

Digital Library

[30]

Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. 2018. HyTE: Hyperplane-based temporally aware knowledge graph embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2001–2011.

[31]

Tingsong Jiang, Tianyu Liu, Tao Ge, Lei Sha, Sujian Li, Baobao Chang, and Zhifang Sui. 2016. Encoding temporal information for time-aware link prediction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2350–2354.

[32]

Rishab Goel, Seyed Mehran Kazemi, Marcus Brubaker, and Pascal Poupart. 2020. Diachronic embedding for temporal knowledge graph completion. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 3988–3995

[33]

Yunpu Ma, Volker Tresp, and Erik A. Daxberger. 2018. Embedding models for episodic knowledge graphs. Journal of Web Semantics. arXiv:1807.00228. Retrieved from https://arxiv.org/abs/1807.00228.

[34]

Timoth´ee Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. In Proceedings of the International Conference on Learning Representations.

[35]

Alberto García-Durán, Sebastijan Dumančić, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 4816–4821.

[36]

Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart. 2020. Relational representation learning for dynamic (knowledge) graphs: A survey. arXiv:1905.11485. Retrieved from https://arxiv.org/pdf/1905.11485v1.pdf.

[37]

Veličković Petar, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. In Proceedings of the International Conference on Learning Representations.

[38]

Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. 2020. DySAT: Deep neural representation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining. 519–527.

Digital Library

[39]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st Conference on Neural Information Processing Systems.

Digital Library

[40]

Kyunghyun Cho, Bart van Merrienboer, C¸ aglar G¨ulc¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 5998–6008.

[41]

Srivastava, Nitish, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929–1958.

[42]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https://arxiv.org/abs/1412.6980.

[43]

Ruffinelli Daniel, Samuel Broscheit, and Rainer Gemulla. 2019. You CAN teach an old dog new tricks! On training knowledge graph embeddings. In Proceedings of the International Conference on Learning Representations.

[44]

Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2013. YAGO3: A knowledge base from multilingual wikipedias. In Proceedings of the 7th Biennial Conference on Innovative Data Systems Research.

[45]

Fredo Erxleben, Michael Günther, Markus Krötzsch, Julian Mendez, and Denny Vrandeăić. 2014. Introducing wikidata to the linked data web. In Proceedings of the 13th International Semantic Web Conference. 50–65.

Digital Library

Cited By

Li XZhou HYao WLi WLiu BLin Y(2024)Intricate Spatiotemporal Dependency Learning for Temporal Knowledge Graph ReasoningACM Transactions on Knowledge Discovery from Data10.1145/364836618:6(1-19)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3648366
Bai LChen MXiao Q(2024)Multi-hop temporal knowledge graph reasoning with multi-agent reinforcement learningApplied Soft Computing10.1016/j.asoc.2024.111727160:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111727
Dong HWang PXiao MNing ZWang PZhou Y(2024)Temporal inductive path neural network for temporal knowledge graph reasoningArtificial Intelligence10.1016/j.artint.2024.104085329:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.artint.2024.104085
Show More Cited By

Index Terms

TPmod: A Tendency-Guided Prediction Model for Temporal Knowledge Graph Completion
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Reasoning about belief and knowledge
  2. Machine learning
    1. Machine learning approaches
      1. Learning latent representations

Recommendations

Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Knowledge Graph (KG) reasoning that predicts missing facts for incomplete KGs has been widely explored. However, reasoning over Temporal KG (TKG) that predicts facts in the future is still far from resolved. The key to predict future facts is to ...
Hyperplane-based time-aware knowledge graph embedding for temporal knowledge graph completion

Knowledge Graph (KG) embedding approaches have been proved effective to infer new facts for a KG based on the existing ones–a problem known as KG completion. However, most of them have focused on static KGs, in fact, relational facts in KGs often show ...
A relation enhanced model for temporal knowledge graph alignment
Abstract
Entity alignment (EA) aims to find entities that point to the same object in multiple knowledge graphs (KGs), i.e., finding equivalent entity pairs. In recent years, with the emergence of temporal knowledge graphs (TKGs), the TKG alignment has ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 15, Issue 3

June 2021

533 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3454120

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2021

Accepted: 01 December 2020

Revised: 01 November 2020

Received: 01 July 2020

Published in TKDD Volume 15, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China
Natural Science Foundation of Hebei Province
Natural Science Foundation of Liaoning Province
Key Project of Scientific Research Funds in Colleges and Universities of Hebei Education Department
Fundamental Research Funds for the Central Universities
Program for 333 Talents in Hebei Province

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
725
Total Downloads

Downloads (Last 12 months)100
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li XZhou HYao WLi WLiu BLin Y(2024)Intricate Spatiotemporal Dependency Learning for Temporal Knowledge Graph ReasoningACM Transactions on Knowledge Discovery from Data10.1145/364836618:6(1-19)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3648366
Bai LChen MXiao Q(2024)Multi-hop temporal knowledge graph reasoning with multi-agent reinforcement learningApplied Soft Computing10.1016/j.asoc.2024.111727160:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111727
Dong HWang PXiao MNing ZWang PZhou Y(2024)Temporal inductive path neural network for temporal knowledge graph reasoningArtificial Intelligence10.1016/j.artint.2024.104085329:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.artint.2024.104085
Li LLiu WXiong ZWang Y(2023)Sequence-Based Modeling for Temporal Knowledge Graph Link PredictionArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44216-2_45(550-562)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44216-2_45
He PZhou GLiu HXia YWang L(2022)Hyperplane-based time-aware knowledge graph embedding for temporal knowledge graph completionJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21195042:6(5457-5469)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/JIFS-211950
Wang JWang BGao JHu YYin B(2022)Multi-Concept Representation Learning for Knowledge Graph CompletionACM Transactions on Knowledge Discovery from Data10.1145/353301717:1(1-19)Online publication date: 30-Apr-2022
https://dl.acm.org/doi/10.1145/3533017
Chen XDing LXiang Y(2021)Neighborhood aggregation based graph attention networks for open-world knowledge graph reasoningJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21188941:2(3797-3808)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.3233/JIFS-211889

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents