Article

MMKG: Multi-modal Knowledge Graphs

Authors:

Alberto Garcia-Duran,

Mathias Niepert,

Daniel Onoro-Rubio,

David S. RosenblumAuthors Info & Claims

The Semantic Web: 16th International Conference, ESWC 2019, Portorož, Slovenia, June 2–6, 2019, Proceedings

Pages 459 - 474

https://doi.org/10.1007/978-3-030-21348-0_30

Published: 02 June 2019 Publication History

Abstract

We present Mmkg, a collection of three knowledge graphs that contain both numerical features and (links to) images for all entities as well as entity alignments between pairs of KGs. Therefore, multi-relational link prediction and entity matching communities can benefit from this resource. We believe this data set has the potential to facilitate the development of novel multi-modal learning approaches for knowledge graphs. We validate the utility of Mmkg in the link prediction task with an extensive set of experiments. These experiments show that the task at hand benefits from learning of multiple feature types.

References

[1]

Achichi, M., et al.: Results of the ontology alignment evaluation initiative 2016. In: OM: Ontology Matching, pp. 73–129 (2016). No commercial editor

[2]

Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z, et al. Aberer K et al. DBpedia: a nucleus for a web of open data The Semantic Web 2007 Heidelberg Springer 722-735

Digital Library

[3]

Bollacker, K.D., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD Conference, pp. 1247–1250 (2008)

[4]

Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS, pp. 2787–2795 (2013)

[5]

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255. IEEE (2009)

[6]

Dredze, M., McNamee, P., Rao, D., Gerber, A., Finin, T.: Entity disambiguation for knowledge base population. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 277–285. Association for Computational Linguistics (2010)

[7]

Galárraga L, Teflioudi C, Hose K, and Suchanek FM Fast rule mining in ontological knowledge bases with AMIE+ VLDB J. 2015 24 6 707-730

Digital Library

[8]

Galárraga, L.A., Preda, N., Suchanek, F.M.: Mining rules to align knowledge bases. In: Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, pp. 43–48. ACM (2013)

[9]

Garcia-Duran, A., Niepert, M.: KBLRN: end-to-end learning of knowledge base representations with latent, relational, and numerical features. In: Uncertainty in Artificial Intelligence Proceedings of the 34th Conference (2018)

[10]

Gardner, M., Mitchell, T.M.: Efficient and expressive knowledge base completion using subgraph feature extraction. In: EMNLP, pp. 1488–1498 (2015)

[11]

Klyne, G., Carroll, J.J., McBride, B.: Resource description framework (RDF): concepts and abstract syntax. W3C Recommendation, February 2004

[12]

Krishna, R., et al.: Visual genome: connecting language and vision using crowdsourced dense image annotations (2016). https://arxiv.org/abs/1602.07332

[13]

Lacoste-Julien, S., Palla, K., Davies, A., Kasneci, G., Graepel, T., Ghahramani, Z.: SiGMa: simple greedy matching for aligning large knowledge bases. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 572–580. ACM (2013)

[14]

Lao, N., Mitchell, T., Cohen, W.W.: Random walk inference and learning in a large scale knowledge base. In: EMNLP, pp. 529–539 (2011)

[15]

Nickel M, Murphy K, Tresp V, and Gabrilovich E A review of relational machine learning for knowledge graphs Proc. IEEE 2016 104 1 11-33

[16]

Oñoro-Rubio, D., Niepert, M., García-Durán, A., González-Sánchez, R., López-Sastre, R.J.: Representation learning for visual-relational knowledge graphs. arXiv preprint arXiv:1709.02314 (2017)

[17]

Pezeshkpour, P., Chen, L., Singh, S.: Embedding multimodal relational data for knowledge base completion. In: EMNLP (2018)

[18]

Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 74–84 (2013)

[19]

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

[20]

Suchanek FM, Abiteboul S, and Senellart P PARIS: probabilistic alignment of relations, instances, and schema Proc. VLDB Endow. 2011 5 3 157-168

Digital Library

[21]

Suchanek, F.M., Kasneci, G., Weikum, G.: YAGO: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web, pp. 697–706. ACM (2007)

[22]

The HDF Group: Hierarchical Data Format, version 5 (1997-NNNN). http://www.hdfgroup.org/HDF5/

[23]

Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57–66 (2015)

[24]

Wu, Q., Wang, P., Shen, C., Dick, A., van den Hengel, A.: Ask me anything: free-form visual question answering based on knowledge from external sources. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4622–4630 (2016)

Cited By

Wang SZhu YLiu HZheng ZChen CLi J(2024)Knowledge Editing for Large Language Models: A SurveyACM Computing Surveys10.1145/369859057:3(1-37)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3698590
Liu ZFang GWang WLiu Q(2024)Multimodal Sign Language Knowledge Graph and Representation: Text Video KeyFrames and Motion TrajectoriesProceedings of the 6th ACM International Conference on Multimedia in Asia10.1145/3696409.3700208(1-7)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3696409.3700208
Xu YLi YXu MZhu ZZhao Y(2024)HKA: A Hierarchical Knowledge Alignment Framework for Multimodal Knowledge Graph CompletionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366428820:8(1-19)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1145/3664288
Show More Cited By

Recommendations

Collective Multi-type Entity Alignment Between Knowledge Graphs
WWW '20: Proceedings of The Web Conference 2020

Knowledge graph (e.g. Freebase, YAGO) is a multi-relational graph representing rich factual information among entities of various types. Entity alignment is the key step towards knowledge graph integration from multiple sources. It aims to identify ...
Multi-modal knowledge graphs representation learning via multi-headed self-attention
Abstract
Traditional knowledge graphs (KG) representation learning focuses on the link information between entities, and the effectiveness of learning is influenced by the complexity of KGs. Considering a multi-modal knowledge graph (MKG), due ...
Highlights
- New dataset containing the multi-modal features of each entity is generated.
- ...
MMEA: Entity Alignment for Multi-modal Knowledge Graph
Knowledge Science, Engineering and Management
Abstract
Entity alignment plays an essential role in the knowledge graph (KG) integration. Though large efforts have been made on exploring the association of relational embeddings between different knowledge graphs, they may fail to effectively describe ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

The Semantic Web: 16th International Conference, ESWC 2019, Portorož, Slovenia, June 2–6, 2019, Proceedings

Jun 2019

615 pages

ISBN:978-3-030-21347-3

DOI:10.1007/978-3-030-21348-0

Editors:
Pascal Hitzler
https://ror.org/04qk6pt94Wright State University, Dayton, OH, USA
,
Miriam Fernández
https://ror.org/05mzfcs16KMi, The Open University, Milton Keynes, UK
,
Krzysztof Janowicz
University of California, Santa Barbara, CA, USA
,
Amrapali Zaveri
https://ror.org/02jz4aj89Maastricht University, Maastricht, The Netherlands
,
Alasdair J.G. Gray
https://ror.org/04mghma93Heriot-Watt University, Edinburgh, UK
,
Vanessa Lopez
IBM Research, Dublin, Ireland
,
Armin Haller
The Australian National University, Canberra, ACT, Australia
,
Karl Hammar
https://ror.org/03t54am93Jönköping University, Jönköping, Sweden

© Springer Nature Switzerland AG 2019.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 02 June 2019

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang SZhu YLiu HZheng ZChen CLi J(2024)Knowledge Editing for Large Language Models: A SurveyACM Computing Surveys10.1145/369859057:3(1-37)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3698590
Liu ZFang GWang WLiu Q(2024)Multimodal Sign Language Knowledge Graph and Representation: Text Video KeyFrames and Motion TrajectoriesProceedings of the 6th ACM International Conference on Multimedia in Asia10.1145/3696409.3700208(1-7)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3696409.3700208
Xu YLi YXu MZhu ZZhao Y(2024)HKA: A Hierarchical Knowledge Alignment Framework for Multimodal Knowledge Graph CompletionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366428820:8(1-19)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1145/3664288
Tang WWang YGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Multi-modal Entity Alignment via Position-enhanced Multi-label PropagationProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658085(366-375)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658085
Yang XGong XTang BLei YDeng YOuyang HZhao GLuo LFeng YDuan BLi SXu YSerra ESpezzano F(2024)CAG: A Consistency-Adaptive Text-Image Alignment Generation for Joint Multimodal Entity-Relation ExtractionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679883(4183-4187)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679883
Zha ZWang JLi ZZhu XSong WXiao YSerra ESpezzano F(2024)M2ConceptBase: A Fine-Grained Aligned Concept-Centric Multimodal Knowledge BaseProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679852(3113-3123)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679852
Li QLi JWu JPeng XJi CPeng HWang LYu P(2024)Triplet-aware graph neural networks for factorized multi-modal knowledge graph entity alignmentNeural Networks10.1016/j.neunet.2024.106479179:COnline publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106479
Chen JYang LWang ZGong M(2024)Higher-order GNN with Local Inflation for entity alignmentKnowledge-Based Systems10.1016/j.knosys.2024.111634293:COnline publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111634
Chen WLi XLong SLei JLi SZhang J(2024)Feature Balance Method for Multi-modal Entity AlignmentPattern Recognition10.1007/978-3-031-78186-5_5(65-80)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/978-3-031-78186-5_5
Du ZYang JWang JHuang YWang L(2024)SPK: Semantic and Positional Knowledge for Zero-Shot Referring Expression ComprehensionPattern Recognition10.1007/978-3-031-78113-1_19(280-295)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/978-3-031-78113-1_19
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten