research-article

Boosting the Speed of Entity Alignment 10 ×: Dual Attention Matching Network with Normalized Hard Sample Mining

Authors:

Man LanAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

Pages 821 - 832

https://doi.org/10.1145/3442381.3449897

Published: 03 June 2021 Publication History

Abstract

Seeking the equivalent entities among multi-source Knowledge Graphs (KGs) is the pivotal step to KGs integration, also known as entity alignment (EA). However, most existing EA methods are inefficient and poor in scalability. A recent summary points out that some of them even require several days to deal with a dataset containing 200,000 nodes (DWY100K). We believe over-complex graph encoder and inefficient negative sampling strategy are the two main reasons. In this paper, we propose a novel KG encoder — Dual Attention Matching Network (Dual-AMN), which not only models both intra-graph and cross-graph information smartly, but also greatly reduces computational complexity. Furthermore, we propose the Normalized Hard Sample Mining Loss to smoothly select hard negative samples with reduced loss shift. The experimental results on widely used public datasets indicate that our method achieves both high accuracy and high efficiency. On DWY100K, the whole running process of our method could be finished in 1,100 seconds, at least 10 × faster than previous work. The performances of our method also outperform previous works across all datasets, where Hits@1 and MRR have been improved from 6% to 13%.

References

[1]

Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary G. Ives. 2007. DBpedia: A Nucleus for a Web of Open Data. In The Semantic Web, 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11-15, 2007(Lecture Notes in Computer Science, Vol. 4825), Karl Aberer, Key-Sun Choi, Natasha Fridman Noy, Dean Allemang, Kyung-Il Lee, Lyndon J. B. Nixon, Jennifer Golbeck, Peter Mika, Diana Maynard, Riichiro Mizoguchi, Guus Schreiber, and Philippe Cudré-Mauroux (Eds.). Springer, 722–735. https://doi.org/10.1007/978-3-540-76298-0_52

Digital Library

[2]

Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger (Eds.). 2787–2795. http://papers.nips.cc/paper/5071-translating-embeddings-for-modeling-multi-relational-data

Digital Library

[3]

Yixin Cao, Zhiyuan Liu, Chengjiang Li, Zhiyuan Liu, Juanzi Li, and Tat-Seng Chua. 2019. Multi-Channel Graph Neural Network for Entity Alignment. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. 1452–1461. https://doi.org/10.18653/v1/p19-1140

[4]

Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. 2017. Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017. 1511–1517. https://doi.org/10.24963/ijcai.2017/209

[5]

Lingbing Guo, Zequn Sun, and Wei Hu. 2019. Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA. 2505–2514. http://proceedings.mlr.press/v97/guo19c.html

[6]

Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality Reduction by Learning an Invariant Mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), 17-22 June 2006, New York, NY, USA. 1735–1742. https://doi.org/10.1109/CVPR.2006.100

Digital Library

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015. IEEE Computer Society, 1026–1034. https://doi.org/10.1109/ICCV.2015.123

Digital Library

[8]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015. 448–456. http://proceedings.mlr.press/v37/ioffe15.html

Digital Library

[9]

Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907(2016). arxiv:1609.02907http://arxiv.org/abs/1609.02907

[10]

Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word translation without parallel data. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=H196sainb

[11]

Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, and Tat-Seng Chua. 2019. Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. 2723–2732. https://doi.org/10.18653/v1/D19-1274

[12]

Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. 2019. Graph Matching Networks for Learning the Similarity of Graph Structured Objects. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA. 3835–3845. http://proceedings.mlr.press/v97/li19d.html

[13]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA. 2181–2187. http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9571

Digital Library

[14]

Xin Mao, Wenting Wang, Huimin Xu, Man Lan, and Yuanbin Wu. 2020. MRAEA: An Efficient and Robust Entity Alignment Approach for Cross-lingual Knowledge Graph. In WSDM ’20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020. 420–428. https://doi.org/10.1145/3336191.3371804

Digital Library

[15]

Xin Mao, Wenting Wang, Huimin Xu, Yuanbin Wu, and Man Lan. 2020. Relational Reflection Entity Alignment. In CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020. 1095–1104. https://doi.org/10.1145/3340531.3412001

Digital Library

[16]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A unified embedding for face recognition and clustering. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015. IEEE Computer Society, 815–823. https://doi.org/10.1109/CVPR.2015.7298682

[17]

Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. 2016. Deep Metric Learning via Lifted Structured Feature Embedding. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. 4004–4012. https://doi.org/10.1109/CVPR.2016.434

[18]

Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Highway Networks. CoRR abs/1505.00387(2015). arxiv:1505.00387http://arxiv.org/abs/1505.00387

[19]

Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8-12, 2007, Carey L. Williamson, Mary Ellen Zurko, Peter F. Patel-Schneider, and Prashant J. Shenoy (Eds.). ACM, 697–706. https://doi.org/10.1145/1242572.1242667

Digital Library

[20]

Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. 2020. Circle Loss: A Unified Perspective of Pair Similarity Optimization. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. 6397–6406. https://doi.org/10.1109/CVPR42600.2020.00643

[21]

Zequn Sun, Wei Hu, and Chengkai Li. 2017. Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding. In The Semantic Web - ISWC 2017 - 16th International Semantic Web Conference, Vienna, Austria, October 21-25, 2017, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 10587), Claudia d’Amato, Miriam Fernández, Valentina A. M. Tamma, Freddy Lécué, Philippe Cudré-Mauroux, Juan F. Sequeda, Christoph Lange, and Jeff Heflin (Eds.). Springer, 628–644. https://doi.org/10.1007/978-3-319-68288-4_37

[22]

Zequn Sun, Wei Hu, Qingheng Zhang, and Yuzhong Qu. 2018. Bootstrapping Entity Alignment with Knowledge Graph Embedding. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. 4396–4402. https://doi.org/10.24963/ijcai.2018/611

[23]

Zequn Sun, JiaCheng Huang, Wei Hu, Muchao Chen, Lingbing Guo, and Yuzhong Qu. 2020. TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs. CoRR abs/2004.13579(2020). arxiv:2004.13579https://arxiv.org/abs/2004.13579

[24]

Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei Zhang, and Yuzhong Qu. 2020. Knowledge Graph Alignment Network with Gated Multi-Hop Neighborhood Aggregation. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. 222–229. https://aaai.org/ojs/index.php/AAAI/article/view/5354

[25]

Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. ArnetMiner: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, August 24-27, 2008, Ying Li, Bing Liu, and Sunita Sarawagi (Eds.). ACM, 990–998. https://doi.org/10.1145/1401890.1402008

Digital Library

[26]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. https://openreview.net/forum?id=rJXMpikCZ

[27]

Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. 2018. Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. 349–357. https://doi.org/10.18653/v1/d18-1032

[28]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Québec City, Québec, Canada. 1112–1119. http://www.aaai.org/ocs/index.php/AAAI/AAAI14/paper/view/8531

[29]

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Rui Yan, and Dongyan Zhao. 2019. Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. 5278–5284. https://doi.org/10.24963/ijcai.2019/733

[30]

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, and Dongyan Zhao. 2019. Jointly Learning Entity and Relation Representations for Entity Alignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. 240–249. https://doi.org/10.18653/v1/D19-1023

[31]

Kun Xu, Liwei Wang, Mo Yu, Yansong Feng, Yan Song, Zhiguo Wang, and Dong Yu. 2019. Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. 3156–3161. https://doi.org/10.18653/v1/p19-1304

[32]

Hsiu-Wei Yang, Yanyan Zou, Peng Shi, Wei Lu, Jimmy Lin, and Xu Sun. 2019. Aligning Cross-Lingual Entities with Multi-Aspect Information. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. 4430–4440. https://doi.org/10.18653/v1/D19-1451

[33]

Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. 2019. Graph Transformer Networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada. 11960–11970. http://papers.nips.cc/paper/9367-graph-transformer-networks

[34]

X. Zhao, W. Zeng, J. Tang, W. Wang, and F. Suchanek. 2020. An Experimental Study of State-of-the-Art Entity Alignment Approaches. IEEE Transactions on Knowledge and Data Engineering (2020), 1–1.

[35]

Qiannan Zhu, Xiaofei Zhou, Jia Wu, Jianlong Tan, and Li Guo. 2019. Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. 1943–1949. https://doi.org/10.24963/ijcai.2019/269

Cited By

Zeng KJin HLv XZhu FHou LZhang YPang FQi YLiu DLi JFeng L(2024)XLORE 3: A Large-Scale Multilingual Knowledge Graph from Heterogeneous Wiki Knowledge ResourcesACM Transactions on Information Systems10.1145/366052142:6(1-47)Online publication date: 19-Aug-2024
https://dl.acm.org/doi/10.1145/3660521
Zhang YWu JYu KWu X(2024)Diverse Structure-Aware Relation Representation in Cross-Lingual Entity AlignmentACM Transactions on Knowledge Discovery from Data10.1145/363877818:4(1-23)Online publication date: 13-Feb-2024
https://dl.acm.org/doi/10.1145/3638778
Jiang XXu CShen YWang YSu FShi ZSun FLi ZGuo JShen HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Toward Practical Entity Alignment Method Design: Insights from New Highly Heterogeneous Knowledge Graph DatasetsProceedings of the ACM Web Conference 202410.1145/3589334.3645720(2325-2336)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645720
Show More Cited By

Recommendations

Relational Reflection Entity Alignment
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Entity alignment aims to identify equivalent entity pairs from different Knowledge Graphs (KGs), which is essential in integrating multi-source KGs. Recently, with the introduction of GNNs into entity alignment, the architectures of recent models have ...
An Entity Alignment Method Based on Graph Attention Network with Pre-classification
Web Information Systems and Applications
Abstract
Entity alignment is the process of identifying entities that point to the same object in different knowledge graphs. Entity alignment is a key step in building knowledge graphs, and the result of entity alignment directly affects the quality of ...
Generative adversarial network for unsupervised multi-lingual knowledge graph entity alignment
Abstract
Entity alignment is an essential process in knowledge graph (KG) fusion, which aims to link entities representing the same real-world object in different KGs, to achieve entity expansion and graph fusion. Recently, embedding-based entity pair ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

50
Total Citations
View Citations
443
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)8

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zeng KJin HLv XZhu FHou LZhang YPang FQi YLiu DLi JFeng L(2024)XLORE 3: A Large-Scale Multilingual Knowledge Graph from Heterogeneous Wiki Knowledge ResourcesACM Transactions on Information Systems10.1145/366052142:6(1-47)Online publication date: 19-Aug-2024
https://dl.acm.org/doi/10.1145/3660521
Zhang YWu JYu KWu X(2024)Diverse Structure-Aware Relation Representation in Cross-Lingual Entity AlignmentACM Transactions on Knowledge Discovery from Data10.1145/363877818:4(1-23)Online publication date: 13-Feb-2024
https://dl.acm.org/doi/10.1145/3638778
Jiang XXu CShen YWang YSu FShi ZSun FLi ZGuo JShen HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Toward Practical Entity Alignment Method Design: Insights from New Highly Heterogeneous Knowledge Graph DatasetsProceedings of the ACM Web Conference 202410.1145/3589334.3645720(2325-2336)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645720
Yang ZDing MHuang TCen YSong JXu BDong YTang J(2024)Does Negative Sampling Matter? a Review With Insights Into its Theory and ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337147346:8(5692-5711)Online publication date: Aug-2024
https://doi.org/10.1109/TPAMI.2024.3371473
Zeng WZhao XTang JFan C(2024)Knowledge Graph Alignment Under Scarce Supervision: A General Framework With Active Cross-View Contrastive LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.332190035:9(11692-11705)Online publication date: Sep-2024
https://doi.org/10.1109/TNNLS.2023.3321900
Liu CChen CLou JTsai MHuang JWang C(2024)SARA: Semantic-assisted Reinforced Active Learning for Entity Alignment2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650292(1-10)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650292
Huang PZhang MZhong ZChai CFan J(2024)Representation Learning for Entity Alignment in Knowledge Graph: A Design Space Exploration2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00267(3462-3475)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00267
Xu BLu YSu BYan X(2024)Position-Aware Active Learning for Multi-Modal Entity AlignmentICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447624(8215-8219)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447624
Li JLu YQin JChen JPan W(2024)Entity Alignment Based on Cross-Graph and Enhanced Attention2024 7th International Conference on Artificial Intelligence and Big Data (ICAIBD)10.1109/ICAIBD62003.2024.10604492(117-123)Online publication date: 24-May-2024
https://doi.org/10.1109/ICAIBD62003.2024.10604492
Cao HMiao HWang WLi LPeng WZhao T(2024)Bilingual phrase induction with local hard negative samplingCAAI Transactions on Intelligence Technology10.1049/cit2.12383Online publication date: Oct-2024
https://doi.org/10.1049/cit2.12383
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents