Abstract
Knowledge Graph Completion (KGC) aims to predict missing entities or relations in knowledge graph but it becomes computationally expensive as KG scales. Existing research focuses on bilinear pooling-based factorization methods (LowFER, TuckER) to solve this problem. These approaches introduce too many trainable parameters which obstruct the deployment of these techniques in many real-world scenarios. In this paper, we introduce a novel parameter-efficient framework, KGRP which a) approximates bilinear pooling using Kernelized Random Projection matrix b) employs CNN for the better fusion of entities and relations to infer missing links. Our experimental results show that KGRP has 73% fewer parameters as compared to the state-of-the-art approaches (LowFER, TuckER) for the knowledge graph completion task while retaining 88% performance for the best baseline. Furthermore, we also provide novel insights on the interpretability of relation embeddings. We also test the effectiveness of KGRP on a large-scale recruitment knowledge graph of 0.25M entities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We will use the terms ‘Knowledge Graph Completion’ and ‘Link Prediction’ interchangeably in the manuscript.
References
Achlioptas, D.: Database-friendly random projections: johnson-lindenstrauss with binary coins. J. Comput. Syst. Sci. 66(4), 671–687 (2003)
Amin, S., Varanasi, S., Dunfield, K.A., Neumann, G.: LowFER: low-rank bilinear pooling for link prediction. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 257–268. PMLR, 13–18 July 2020. http://proceedings.mlr.press/v119/amin20a.html
Amin, S., Varanasi, S., Dunfield, K.A., Neumann, G.: Lowfer: low-rank bilinear pooling for link prediction. In: International Conference on Machine Learning, pp. 257–268. PMLR (2020)
Balazevic, I., Allen, C., Hospedales, T.: Tucker: tensor factorization for knowledge graph completion. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019). https://doi.org/10.18653/v1/d19-1522, http://dx.doi.org/10.18653/v1/D19-1522
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250. SIGMOD ’08, Association for Computing Machinery, New York, NY, USA (2008). https://doi.org/10.1145/1376616.1376746
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Neural Information Processing Systems (NIPS), pp. 1–9. South Lake Tahoe, United States, December 2013. https://hal.archives-ouvertes.fr/hal-00920777
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Process. Syst. 26 (2013)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2d knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018). https://ojs.aaai.org/index.php/AAAI/article/view/11573
Fukui, A., Park, D.H., Yang, D., Rohrbach, A., Darrell, T., Rohrbach, M.: Multimodal compact bilinear pooling for visual question answering and visual grounding. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 457–468. Association for Computational Linguistics, Austin, Texas, November 2016. https://doi.org/10.18653/v1/D16-1044, https://www.aclweb.org/anthology/D16-1044
Gao, Y., Beijbom, O., Zhang, N., Darrell, T.: Compact bilinear pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 317–326 (2016)
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (volume 1: Long papers), pp. 687–696 (2015)
Li, P., Hastie, T.J., Church, K.W.: Very sparse random projections. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 287–296. KDD ’06, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1150402.1150436
Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1457 (2015)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)
López-Sánchez, D., Arrieta, A.G., Corchado, J.M.: Data-independent random projections from the feature-space of the homogeneous polynomial kernel. Pattern Recognit. 82, 130–146 (2018)
López-Sánchez, D., Arrieta, A.G., Corchado, J.M.: Compact bilinear pooling via kernelized random projection for fine-grained image categorization on low computational power devices. Neurocomputing 398, 411–421 (2020)
Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995). https://doi.org/10.1145/219717.219748
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: knowledge graph embedding by relational rotation in complex space (2019)
Taheri, S., Toygar, Ö.: On the use of DAG-CNN architecture for age estimation with multi-stage features fusion. Neurocomputing 329, 300–310 (2019)
Tenenbaum, J.B., Freeman, W.T.: Separating style and content with bilinear models. Neural Comput. 12(6), 1247–1283 (2000)
Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M.: Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1499–1509. Association for Computational Linguistics, Lisbon, Portugal, September 2015. https://doi.org/10.18653/v1/D15-1174, https://www.aclweb.org/anthology/D15-1174
Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: Balcan, M.F., Weinberger, K.Q. (eds.) Proceedings of The 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 48, pp. 2071–2080. PMLR, New York, New York, USA, 20–22 June 2016. http://proceedings.mlr.press/v48/trouillon16.html
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)
West, R., Gabrilovich, E., Murphy, K., Sun, S., Gupta, R., Lin, D.: Knowledge base completion via search-based question answering. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 515–526. WWW ’14, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2566486.2568032
Yang, B., Yih, S.W.t., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the International Conference on Learning Representations (ICLR) 2015. ICLR (2015). https://www.microsoft.com/en-us/research/publication/embedding-entities-and-relations-for-learning-and-inference-in-knowledge-bases/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Goyal, N., Goel, A., Garg, T., Sachdeva, N., Kumaraguru, P. (2024). Efficient Knowledge Graph Embeddings via Kernelized Random Projections. In: Sachdeva, S., Watanobe, Y. (eds) Big Data Analytics in Astronomy, Science, and Engineering. BDA 2023. Lecture Notes in Computer Science, vol 14516. Springer, Cham. https://doi.org/10.1007/978-3-031-58502-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-58502-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-58501-2
Online ISBN: 978-3-031-58502-9
eBook Packages: Computer ScienceComputer Science (R0)