research-article

Open access

SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs

Authors:

Evgeny Kharlamov,

Jie TangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 860 - 870

https://doi.org/10.1145/3485447.3511945

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing Web-scale KGs. Over the course of its development, the label supervision has been considered necessary for accurate alignments. Inspired by the recent progress of self-supervised learning, we explore the extent to which we can get rid of supervision for entity alignment. Commonly, the label information (positive entity pairs) is used to supervise the process of pulling the aligned entities in each positive pair closer. However, our theoretical analysis suggests that the learning of entity alignment can actually benefit more from pushing unlabeled negative pairs far away from each other than pulling labeled positive pairs close. By leveraging this discovery, we develop the self-supervised learning objective for entity alignment. We present SelfKG with efficient strategies to optimize this objective for aligning entities without label supervision. Extensive experiments on benchmark datasets demonstrate that SelfKG without supervision can match or achieve comparable results with state-of-the-art supervised baselines. The performance of SelfKG suggests that self-supervised learning offers great potential for entity alignment in KGs. The code and data are available at https://github.com/THUDM/SelfKG.

References

[1]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS. 1–9.

[2]

Yixin Cao, Zhiyuan Liu, Chengjiang Li, Juanzi Li, and Tat-Seng Chua. 2019. Multi-Channel Graph Neural Network for Entity Alignment. In ACL. 1452–1461.

[3]

Bo Chen, Jing Zhang, Xiaobin Tang, Hong Chen, and Cuiping Li. 2020. JarKA: Modeling Attribute Interactions for Cross-lingual Knowledge Alignment. In PAKDD. Springer.

[4]

Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. 2017. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In IJCAI.

[5]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In ICML. PMLR, 1597–1607.

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. 4171–4186.

[7]

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2021. All nlp tasks are generation tasks: A general pretraining framework. arXiv preprint arXiv:2103.10360(2021).

[8]

Jeffrey Scott Eder. 2012. Knowledge graph based search system. US Patent App. 13/404,109.

[9]

Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852(2020).

[10]

Matthias Fey, Jan E Lenssen, Christopher Morris, Jonathan Masci, and Nils M Kriege. 2020. Deep Graph Matching Consensus. In ICLR.

[11]

Lingbing Guo, Zequn Sun, and Wei Hu. 2019. Learning to exploit long-term relational dependencies in knowledge graphs. In ICML. PMLR, 2505–2514.

[12]

Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2020. A survey on knowledge graph-based recommender systems. TKDE (2020).

[13]

Michael Gutmann and Aapo Hyvärinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In AIStats. 297–304.

[14]

Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Liang Zhang, Wentao Han, Minlie Huang, 2021. Pre-trained models: Past, present and future. AI Open (2021).

[15]

Yanchao Hao, Yuanzhe Zhang, Shizhu He, Kang Liu, and Jun Zhao. 2016. A joint embedding method for entity alignment of knowledge bases. In CCKS. Springer, 3–14.

[16]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729–9738.

[17]

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. NIPS (2020).

[18]

Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, and Tat-Seng Chua. 2019. Semi-supervised entity alignment via joint knowledge embedding model and cross-graph model. In EMNLP. 2723–2732.

[19]

Feng-Lin Li, Hehong Chen, Guohai Xu, Tian Qiu, Feng Ji, Ji Zhang, and Haiqing Chen. 2020. AliMeKG: Domain Knowledge Graph Construction and Application in E-commerce. In CIKM. 2581–2588.

[20]

Juanzi Li, Jie Tang, Yi Li, and Qiong Luo. 2008. Rimom: A dynamic multistrategy ontology alignment framework. TKDE 21, 8 (2008), 1218–1232.

[21]

Lingli Li, Jianzhong Li, and Hong Gao. 2014. Rule-based method for entity resolution. TKDE 27, 1 (2014), 250–263.

[22]

Xiao Liu, Li Mian, Yuxiao Dong, Fanjin Zhang, Jing Zhang, Jie Tang, Peng Zhang, Jibing Gong, and Kuansan Wang. 2021. OAG_know: Self-supervised Learning for Linking Knowledge Graphs. TKDE (2021).

[23]

Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. 2021. Self-supervised learning: Generative or contrastive. TKDE (2021).

[24]

Heiko Paulheim. 2017. Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web 8, 3 (2017), 489–508.

[25]

Shichao Pei, Lu Yu, Robert Hoehndorf, and Xiangliang Zhang. 2019. Semi-supervised entity alignment via knowledge graph embedding with awareness of degree difference. In WWW. 3130–3136.

[26]

Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. Gcc: Graph contrastive coding for graph neural network pre-training. In SIGKDD. 1150–1160.

Digital Library

[27]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.

[28]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. [n. d.]. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR ([n. d.]).

[29]

Xiaofei Shi and Yanghua Xiao. 2019. Modeling multi-mapping relations for precise cross-lingual entity alignment. In EMNLP. 813–822.

[30]

Zequn Sun, Wei Hu, and Chengkai Li. 2017. Cross-lingual entity alignment via joint attribute-preserving embedding. In ISWC. Springer, 628–644.

[31]

Zequn Sun, Wei Hu, Qingheng Zhang, and Yuzhong Qu. 2018. Bootstrapping Entity Alignment with Knowledge Graph Embedding. In IJCAI, vol.18. 4396–4402.

[32]

Zequn Sun, Jiacheng Huang, Wei Hu, Muhao Chen, Lingbing Guo, and Yuzhong Qu. 2019. Transedge: Translating relation-contextualized embeddings for knowledge graphs. In ISWC. Springer, 612–629.

[33]

Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei Zhang, and Yuzhong Qu. 2020. Knowledge graph alignment network with gated multi-hop neighborhood aggregation. In AAAI, Vol. 34. 222–229.

[34]

Jie Tang, Juanzi Li, Bangyong Liang, Xiaotong Huang, Yi Li, and Kehong Wang. 2006. Using Bayesian decision for ontology mapping. JWS 4, 4 (2006), 243–262.

Digital Library

[35]

Xiaobin Tang, Jing Zhang, Bo Chen, Yang Yang, Hong Chen, and Cuiping Li. 2021. BERT-INT: a BERT-based interaction model for knowledge graph alignment. In IJCAI. 3174–3180.

[36]

Bayu Distiawan Trisedya, Jianzhong Qi, and Rui Zhang. 2019. Entity alignment between knowledge graphs using attribute embeddings. In AAAI, Vol. 33. 297–304.

[37]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.

[38]

Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In ICML. PMLR, 9929–9939.

[39]

Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. 2018. Cross-lingual knowledge graph alignment via graph convolutional networks. In EMNLP. 349–357.

[40]

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Rui Yan, and Dongyan Zhao. 2019. Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs. In IJCAI.

[41]

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, and Dongyan Zhao. 2019. Jointly Learning Entity and Relation Representations for Entity Alignment. In EMNLP. 240–249.

[42]

Kun Xu, Liwei Wang, Mo Yu, Yansong Feng, Yan Song, Zhiguo Wang, and Dong Yu. 2019. Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network. In ACL.

[43]

Hsiu-Wei Yang, Yanyan Zou, Peng Shi, Wei Lu, Jimmy Lin, and Xu Sun. 2019. Aligning Cross-Lingual Entities with Multi-Aspect Information. In EMNLP. 4431–4441.

[44]

Kai Yang, Shaoqin Liu, Junfeng Zhao, Yasha Wang, and Bing Xie. 2020. COTSAE: CO-Training of Structure and Attribute Embeddings for Entity Alignment. In AAAI, Vol. 34. 3025–3032.

[45]

Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. NIPS (2019).

[46]

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In EMNLP.

[47]

Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph contrastive learning with augmentations. NIPS (2020).

[48]

Kaisheng Zeng, Chengjiang Li, Lei Hou, Juanzi Li, and Ling Feng. 2021. A comprehensive survey of entity alignment for knowledge graphs. AI Open 2(2021), 1–13.

[49]

Weixin Zeng, Xiang Zhao, Jiuyang Tang, and Xuemin Lin. 2019. Collective Embedding-based Entity Alignment via Adaptive Features.

[50]

Fanjin Zhang, Xiao Liu, Jie Tang, Yuxiao Dong, Peiran Yao, Jie Zhang, Xiaotao Gu, Yan Wang, Bin Shao, Rui Li, 2019. Oag: Toward linking large-scale heterogeneous entity graphs. In SIGKDD. 2585–2595.

Digital Library

[51]

Jing Zhang, Bo Chen, Xianming Wang, Hong Chen, Cuiping Li, Fengmei Jin, Guojie Song, and Yutao Zhang. 2018. Mego2vec: Embedding matched ego networks for user alignment across social networks. In CIKM. 327–336.

[52]

Qingheng Zhang, Zequn Sun, Wei Hu, Muhao Chen, Lingbing Guo, and Yuzhong Qu. 2019. Multi-view Knowledge Graph Embedding for Entity Alignment. In IJCAI. 5429–5435.

[53]

Yutao Zhang, Jie Tang, Zhilin Yang, Jian Pei, and Philip S Yu. 2015. Cosnet: Connecting heterogeneous social networks with local and global consistency. In SIGKDD. 1485–1494.

Digital Library

[54]

Hao Zhu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2017. Iterative Entity Alignment via Joint Knowledge Embeddings. In IJCAI, Vol. 17. 4258–4264.

[55]

Qiannan Zhu, Xiaofei Zhou, Jia Wu, Jianlong Tan, and Li Guo. 2019. Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs. In IJCAI. 1943–1949.

[56]

Yao Zhu, Hongzhi Liu, Zhonghai Wu, and Yingpeng Du. 2021. Relation-Aware Neighborhood Matching Model for Entity Alignment. In AAAI, Vol. 35. 4749–4756.

Cited By

Huo NCheng RKao BNing WHaldar NLi XLi JNajafi MLi TQu G(2024)ZeroEA: A Zero-Training Entity Alignment Framework via Pre-Trained Language ModelProceedings of the VLDB Endowment10.14778/3654621.365464017:7(1765-1774)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.14778/3654621.3654640
Zeng KJin HLv XZhu FHou LZhang YPang FQi YLiu DLi JFeng L(2024)XLORE 3: A Large-Scale Multilingual Knowledge Graph from Heterogeneous Wiki Knowledge ResourcesACM Transactions on Information Systems10.1145/366052142:6(1-47)Online publication date: 19-Aug-2024
https://dl.acm.org/doi/10.1145/3660521
Xie YLu JHo JNahab FHu XYang CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)PromptLink: Leveraging Large Language Models for Cross-Source Biomedical Concept LinkingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657904(2589-2593)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657904
Show More Cited By

Index Terms

SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Interactive Contrastive Learning for Self-Supervised Entity Alignment
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without the use of pre-aligned entity pairs. The current state-of-the-art (SOTA) self-supervised EA approach draws inspiration from contrastive ...
Similarity propagation based semi-supervised entity alignment
Abstract
Entity alignment aims to identify entities referring to the same real world object among multiple knowledge graphs. Current embedding based approaches suffer from the lack of labeled entity pairs as training data. Some works attempt to boost the ...
Semi-Supervised Entity Alignment via Knowledge Graph Embedding with Awareness of Degree Difference
WWW '19: The World Wide Web Conference

Entity alignment associates entities in different knowledge graphs if they are semantically same, and has been successfully used in the knowledge graph construction and connection. Most of the recent solutions for entity alignment are based on knowledge ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSFC

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
2,957
Total Downloads

Downloads (Last 12 months)635
Downloads (Last 6 weeks)106

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huo NCheng RKao BNing WHaldar NLi XLi JNajafi MLi TQu G(2024)ZeroEA: A Zero-Training Entity Alignment Framework via Pre-Trained Language ModelProceedings of the VLDB Endowment10.14778/3654621.365464017:7(1765-1774)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.14778/3654621.3654640
Zeng KJin HLv XZhu FHou LZhang YPang FQi YLiu DLi JFeng L(2024)XLORE 3: A Large-Scale Multilingual Knowledge Graph from Heterogeneous Wiki Knowledge ResourcesACM Transactions on Information Systems10.1145/366052142:6(1-47)Online publication date: 19-Aug-2024
https://dl.acm.org/doi/10.1145/3660521
Xie YLu JHo JNahab FHu XYang CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)PromptLink: Leveraging Large Language Models for Cross-Source Biomedical Concept LinkingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657904(2589-2593)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657904
Wu DLi TZhao YLiu JTang ZYang Z(2024)A Novel Entity and Relation Joint Interaction Learning Approach for Entity AlignmentInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402450004934:05(821-843)Online publication date: 19-Mar-2024
https://doi.org/10.1142/S0218194024500049
Zeng WZhao XTang JFan C(2024)Knowledge Graph Alignment Under Scarce Supervision: A General Framework With Active Cross-View Contrastive LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.332190035:9(11692-11705)Online publication date: Sep-2024
https://doi.org/10.1109/TNNLS.2023.3321900
Wang YSun HWang JWang JTang WQi QSun SLiao J(2024)Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity Alignment2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00274(3559-3572)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00274
Liang YCai WYang MJiang Y(2024)An unsupervised multi-view contrastive learning framework with attention-based reranking strategy for entity alignmentNeural Networks10.1016/j.neunet.2024.106583179(106583)Online publication date: Nov-2024
https://doi.org/10.1016/j.neunet.2024.106583
Zhang XLiu YWei HShan SZhao Z(2024)A self-supervised entity alignment framework via attribute correctionJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2024.10216736:8(102167)Online publication date: Oct-2024
https://doi.org/10.1016/j.jksuci.2024.102167
Yang ZLi L(2024)Knowledge graph-based recommendation with knowledge noise reduction and data augmentationApplied Intelligence10.1007/s10489-024-05657-x54:21(10333-10359)Online publication date: 13-Aug-2024
https://doi.org/10.1007/s10489-024-05657-x
Zhu BWang RWang JShao FWang K(2024)A survey: knowledge graph entity alignment research based on graph embeddingArtificial Intelligence Review10.1007/s10462-024-10866-457:9Online publication date: 3-Aug-2024
https://doi.org/10.1007/s10462-024-10866-4
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents