Abstract
Graph neural networks (GNNs) have achieved a great success on graph-based tasks. Most existing GNNs pay attention to the message passing process, where features propagate along the topology of the graph. Recently, dimension-wise separable 2-D graph convolution (DSGC) points out that GNNs ignore the effect of interactions between the features, and specially designs a dimension-wise feature affinity matrix via predefined corpuses. Although DSGC further improves the performance, we find that it has two shortcomings. For one thing, it performs poorly on low homophily datasets. For another thing, it tends to be over-smoothing as the number of layers increases, making node embeddings indistinguishable. In our empirical analysis, these two shortcomings are caused by the serial message passing mechanism of topology and features. To solve this problem, we propose a parallel-space graph convolution (PSGC) in this paper. The central idea is that we design a parallel message passing mechanism in dual spaces including a topology space and a feature space, which can adapt to different homophily real-world datasets and alleviate the over-smoothing issue. Besides, we design the dimension-wise feature affinity matrix derived by the original node features instead of corpuses in DSGC to alleviate the severe overfitting in the feature space, where corpuses are not provided by many benchmark datasets. Our extensive experiments on benchmark datasets obviously show that PSGC achieves significant performance gain over state-of-the-art methods for semi-supervised node classification.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bo, D., Wang, X., Shi, C., Shen, H.: Beyond low-frequency information in graph convolutional networks. arXiv preprint arXiv:2101.00797 (2021)
Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., Sun, X.: Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 3438–3445 (2020)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. Adv. Neural Inf. Process. Syst. 29, 3844–3852 (2016)
Feng, W., et al.: Graph random neural networks for semi-supervised learning on graphs. Adv. Neural Inf. Process. Syst. 33, 22092–22103 (2020)
Grover, A., Leskovec, J.: Node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Klicpera, J., Bojchevski, A., Günnemann, S.: Predict then propagate: graph neural networks meet personalized Pagerank. arXiv preprint arXiv:1810.05997 (2018)
Li, G., Muller, M., Thabet, A., Ghanem, B.: Deepgcns: can gcns go as deep as CNNs? In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9267–9276 (2019)
Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-Second AAAI conference on artificial intelligence (2018)
Li, Q., Zhang, X., Liu, H., Dai, Q., Wu, X.M.: Dimensionwise separable 2-D graph convolution for unsupervised and semi-supervised learning on graphs. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 953–963 (2021)
Liu, M., Gao, H., Ji, S.: Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 338–348 (2020)
Pal, S.K., Mitra, S.: Multilayer perceptron, fuzzy sets, classifiaction (1992)
Pei, H., Wei, B., Chang, K.C.C., Lei, Y., Yang, B.: Geom-GCN: geometric graph convolutional networks. arXiv preprint arXiv:2002.05287 (2020)
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)
Rong, Y., Huang, W., Xu, T., Huang, J.: DropEdge: towards deep graph convolutional networks on node classification. arXiv preprint arXiv:1907.10903 (2019)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Wang, X., Zhu, M., Bo, D., Cui, P., Shi, C., Pei, J.: Am-GCN: adaptive multi-channel graph convolutional networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1243–1253 (2020)
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: International Conference on Machine Learning, pp. 6861–6871. PMLR (2019)
Yang, C., Wang, R., Yao, S., Liu, S., Abdelzaher, T.: Revisiting over-smoothing in deep GCNs. arXiv preprint arXiv:2003.13663 (2020)
Zhou, K., Dong, Y., Lee, W.S., Hooi, B., Xu, H., Feng, J.: Effective training strategies for deep graph neural networks. arXiv preprint arXiv:2006.07107 (2020)
Zhu, J., Yan, Y., Zhao, L., Heimann, M., Akoglu, L., Koutra, D.: Beyond homophily in graph neural networks: current limitations and effective designs. arXiv preprint arXiv:2006.11468 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, Z., Yan, H. (2022). Parallel Message Passing in Dual-space on Graphs. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13530. Springer, Cham. https://doi.org/10.1007/978-3-031-15931-2_51
Download citation
DOI: https://doi.org/10.1007/978-3-031-15931-2_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15930-5
Online ISBN: 978-3-031-15931-2
eBook Packages: Computer ScienceComputer Science (R0)