MIED : An Improved Graph Neural Network for Node Embedding in Heterogeneous Graphs

Mingjian  Ni; Yinghao Song; Gongju Wang; Lanxiao Feng; Yang Li; Long Yan; Dazhong Li; Yanfei Wang; Shikun Zhang; Yulun Song

doi:10.4108/eetsis.3824

Authors

Mingjian Ni Peking University
Yinghao Song China Unicom Digital Technology Company Limited
Gongju Wang China Unicom Digital Technology Company Limited
Lanxiao Feng China Unicom Network Communications Company Limited
Yang Li China Unicom Digital Technology Company Limited
Long Yan China Unicom Digital Technology Company Limited
Dazhong Li China Unicom Digital Technology Company Limited
Yanfei Wang Unicom (Beijing) Industrial Internet Company Limited
Shikun Zhang Peking University
Yulun Song China Unicom Network Communications Company Limited

DOI:

https://doi.org/10.4108/eetsis.3824

Keywords:

Heterogeneous Graph, Node Embedding, Metapath, Graph Convolutional Network, Exponential Decay Encoder

Abstract

This paper proposes a Metapath-Infused Exponential Decay graph neural network (MIED) approach for node embedding in heterogeneous graphs. It is designed to address limitations in existing methods, which usually lose the graph information during feature alignment and ignore the different importance of nodes during metapath aggregation. Firstly, graph convolutional network (GCN) is applied on the subgraphs, which is derived from the original graph with given metapaths to transform node features. Secondly, an exponential decay encoder (EDE) is designed, in which the influence of nodes on starting point decays exponentially with a fixed parameter as they move farther away from it. Thirdly, a set of experiments is conducted on two selected datasets of heterogeneous graphs, i.e., IMDb and DBLP, for comparison purposes. The results show that MIED outperforms selected approaches, e.g., GAT, HAN, MAGNN, etc. Thus, our approach is proven to be able to take full advantage of graph information considering node weights based on distance aspects. Finally, relevant parameters are analyzed and the recommended hyperparameter setting is given.

References

Singh, R., Subramani, S., Du, J., Zhang, Y., Wang, H., Miao, Y. and Ahmed, K. (2023) Antisocial behavior identification from twitter feeds using traditional machine learning algorithms and deep learning. EAI Endorsed Transactions on Scalable Information Systems 10(4): e17–e17.

Zhou, Y., Lin, Z., La, Y., Huang, J. and Wang, X. (2022) Analysis and design of power system transformer standard based on knowledge graph. EAI Endorsed Transactions on Scalable Information Systems 10(2).

Kumar, S., Mallik, A., Khetarpal, A. and Panda, B. (2022) Influence maximization in social networks using graph embedding and graph neural network. Information Sciences 607: 1617–1636.

Liu, J., Xia, F., Feng, X., Ren, J. and Liu, H. (2022) Deep graph learning for anomalous citation detection. IEEE Transactions on Neural Networks and Learning Systems 33(6): 2543–2557.

Zhou, Y., Lin, Z., Tu, L., Huang, J. and Zhang, Z. (2022) Analysis and design of standard knowledge service system based on deep learning. EAI Endorsed Transactions on Scalable Information Systems 10(2).

Le, T., Le, N. and Le, B. (2023) Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Systems with Applications 214: 119122.

Zhou, J., Liu, L., Wei, W. and Fan, J. (2022) Network representation learning: from preprocessing, feature extraction to node embedding. ACM Computing Surveys (CSUR) 55(2): 1–35.

Ou, M., Cui, P., Pei, J., Zhang, Z. and Zhu, W. (2016) Asymmetric transitivity preserving graph embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining: 1105–1114.

Cao, S., Lu,W. and Xu, Q. (2015) Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM international on conference on information and knowledge management: 891–900.

Lee, D.D. and Seung, H.S. (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755): 788–791.

Perozzi, B., Al-Rfou, R. and Skiena, S. (2014) Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining: 701–710.

Grover, A. and Leskovec, J. (2016) node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACMSIGKDD international conference on Knowledge discovery and data mining: 855–864.

Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L. et al. (2020) Graph neural networks: A review of methods and applications. AI open 1: 57–81.

Di Natale, R., Ferro, A., Giugno, R., Mongiovì, M., Pulvirenti, A. and Shasha, D. (2010) Sing: Subgraph search in non-homogeneous graphs. BMC bioinformatics 11: 1–15.

Hu, Z., Dong, Y., Wang, K. and Sun, Y. (2020) Heterogeneous graph transformer. In Proceedings of the web conference 2020: 2704–2710.

Darban, Z.Z. and Valipour, M.H. (2022) Ghrs: Graphbased hybrid recommendation system with application to movie recommendation. Expert Systems with Applications 200: 116850.

Fu, X., Zhang, J., Meng, Z. and King, I. (2020) Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020: 2331–2341.

Oghina, A., Breuss, M., Tsagkias, M. and De Rijke, M. (2012) Predicting imdb movie ratings using social media. In ECIR (Springer): 503–507.

Deng, H., King, I. and Lyu, M.R. (2008) Formal models for expert finding on dblp bibliography data. In 2008 Eighth IEEE International Conference on Data Mining (IEEE): 163–172.

Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C. and Philip, S.Y. (2020) A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32(1): 4–24.

Chen, D., Lin, Y., Li,W., Li, P., Zhou, J. and Sun, X. (2020) Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI conference on artificial intelligence, 34: 3438–3445.

Chen, M., Wei, Z., Huang, Z., Ding, B. and Li, Y. (2020) Simple and deep graph convolutional networks. In International conference on machine learning (PMLR): 1725–1735.

Sarki, R., Ahmed, K., Wang, H., Zhang, Y. and Wang, K. (2022) Convolutional neural network for multiclass classification of diabetic eye disease. EAI Endorsed Transactions on Scalable Information Systems 9(4): e5–e5.

Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T. and Weinberger, K. (2019) Simplifying graph convolutional networks. In International conference on machine learning (PMLR): 6861–6871.

Hamilton,W., Ying, Z. and Leskovec, J. (2017) Inductive representation learning on large graphs. Advances in neural information processing systems 30.

Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P. and Bengio, Y. (2017) Graph attention networks. arXiv preprint arXiv:1710.10903 .

Brody, S., Alon, U. and Yahav, E. (2021) How attentive are graph attention networks? arXiv preprint arXiv:2105.14491 .

Wang, X., Bo, D., Shi, C., Fan, S., Ye, Y. and Philip, S.Y. (2022) A survey on heterogeneous graph embedding: methods, techniques, applications and sources. IEEE Transactions on Big Data .

Yin, J., Tang, M., Cao, J., Wang, H., You, M. and Lin, Y. (2022) Vulnerability exploitation time prediction: an integrated framework for dynamic imbalanced learning. World Wide Web : 1–23.

Pandey, D., Wang, H., Yin, X., Wang, K., Zhang, Y. and Shen, J. (2022) Automatic breast lesion segmentation in phase preserved dce-mris. Health Information Science and Systems 10(1): 9.

Dong, Y., Chawla, N.V. and Swami, A. (2017) metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining: 135–144.

Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P. and Yu, P.S. (2019) Heterogeneous graph attention network. In The world wide web conference: 2022–2032.

Linmei, H., Yang, T., Shi, C., Ji, H. and Li, X. (2019) Heterogeneous graph attention networks for semisupervised short text classification. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP): 4821–4830.

Bi, S., Li, Z., Brown, M., Wang, L. and Xu, Y. (2022) Dynamic weighted and heat-map integrated scalable information path-planning algorithm. EAI Endorsed Transactions on Scalable Information Systems 10(2).