Label-Enhanced Graph Neural Network for Semi-Supervised Node Classification
Authors: 
Le Yu, Leilei Sun, Bowen Du, Tongyu Zhu, Weifeng LvAuthors Info & Claims
Le Yu, Leilei Sun, Bowen Du, Tongyu Zhu, Weifeng LvAuthors Info & ClaimsPages 11529 - 11540
Abstract
Graph Neural Networks (GNNs) have been widely applied in the semi-supervised node classification task, where a key point lies in how to sufficiently leverage the limited but valuable label information. Most of the classical GNNs solely use the known labels for computing the classification loss at the output. In recent years, several methods have been designed to additionally utilize the labels at the input. One part of the methods augment the node features via concatenating or adding them with the one-hot encodings of labels, while other methods optimize the graph structure by assuming neighboring nodes tend to have the same label. To bring into full play the rich information of labels, in this article we present a label-enhanced learning framework for GNNs, which first models each label as a virtual center for intra-class nodes and then jointly learns the representations of both nodes and labels. Our approach could not only smooth the representations of nodes belonging to the same class, but also explicitly encode the label semantics into the learning process of GNNs. Moreover, a training node selection technique is provided to eliminate the potential label leakage issue and guarantee the model generalization ability. Finally, an adaptive self-training strategy is proposed to iteratively enlarge the training set with more reliable pseudo labels and distinguish the importance of each pseudo-labeled node during the model training process. Experimental results on both real-world and synthetic datasets demonstrate our approach can not only consistently outperform the state-of-the-arts, but also effectively smooth the representations of intra-class nodes.
References
[1]
Z. Zhang, P. Cui, and W. Zhu, “Deep learning on graphs: A survey,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 1, pp. 249–270, Jan. 2022.
[2]
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proc. 5th Int. Conf. Learn. Representations, 2017.
[3]
W. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C. Hsieh, “Cluster-GCN: An efficient algorithm for training deep and large graph convolutional networks,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2019, pp. 257–266.
[4]
W. L. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 1024–1034.
[5]
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 1, pp. 4–24, Jan. 2021.
[6]
P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in Proc. 6th Int. Conf. Learn. Representations, 2018.
[7]
Y. Wang, “Bag of tricks of semi-supervised classification with graph neural networks,” 2021,.
[8]
Y. Shi, Z. Huang, S. Feng, H. Zhong, W. Wang, and Y. Sun, “Masked label prediction: Unified message passing model for semi-supervised classification,” in Proc. 13th Int. Joint Conf. Artif. Intell., 2021, pp. 1548–1554.
[9]
D. Chen et al., “HighwayGraph: Modelling long-distance node relations for improving general graph neural network,” 2019,.
[10]
H. Yang, X. Yan, X. Dai, Y. Chen, and J. Cheng, “Self-enhanced GNN: Improving graph neural networks using model outputs,” in Proc. IEEE Int. Joint Conf. Neural Netw., 2021, pp. 1–8.
[11]
L. Yang, Z. Kang, X. Cao, D. Jin, B. Yang, and Y. Guo, “Topology optimization based graph convolutional network,” in Proc. 28th Int. Joint Conf. Artif. Intell., 2019, pp. 4054–4061.
[12]
H. Wang and J. Leskovec, “Combining graph convolutional neural networks and label propagation,” ACM Trans. Inf. Syst., vol. 40, no. 4, pp. 1–27, 2021.
[13]
D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf, “Learning with local and global consistency,” in Proc. Adv. Neural Inf. Process. Syst., 2003, pp. 321–328.
[14]
X. Zhu, Z. Ghahramani, and J. D. Lafferty, “Semi-supervised learning using gaussian fields and harmonic functions,” in Proc. Mach. Learn. Proc. 20th Int. Conf., 2003, pp. 912–919.
[15]
J. Zhou, G. Cui, Z. Zhang, C. Yang, Z. Liu, and M. Sun, “Graph neural networks: A review of methods and applications,” 2018,.
[16]
B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: Online learning of social representations,” in Proc. 20th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2014, pp. 701–710.
[17]
J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, “LINE: Large-scale information network embedding,” in Proc. 24th Int. Conf. World Wide Web, 2015, pp. 1067–1077.
[18]
A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2016, pp. 855–864.
[19]
Q. Li, Z. Han, and X. Wu, “Deeper insights into graph convolutional networks for semi-supervised learning,” in Proc. 32nd AAAI Conf. Artif. Intell., 2018, pp. 3538–3545.
[20]
Z. Zhou, J. Shi, S. Zhang, Z. Huang, and Q. Li, “Effective semi-supervised node classification on few-labeled graph data,” 2019,.
[21]
K. Sun, Z. Lin, and Z. Zhu, “Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes,” in Proc. 34th AAAI Conf. Artif. Intell., 2020, pp. 5892–5899.
[22]
C. Sun and G. Wu, “Scalable and adaptive graph neural networks with self-label-enhanced training,” 2021,.
[23]
M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in Proc. 15th Eur. Conf. Comput. Vis., 2018, pp. 139–156.
[24]
Y. Sun and J. Han, “Mining heterogeneous information networks: A structural analysis approach,” SIGKDD Explorations, vol. 14, no. 2, pp. 20–28, 2012.
[25]
C. Shi, Y. Li, J. Zhang, Y. Sun, and P. S. Yu, “A survey of heterogeneous information network analysis,” IEEE Trans. Knowl. Data Eng., vol. 29, no. 1, pp. 17–37, Jan. 2017.
[26]
X. Wang et al., “Heterogeneous graph attention network,” in Proc. World Wide Web Conf., 2019, pp. 2022–2032.
[27]
C. Zhang, D. Song, C. Huang, A. Swami, and N. V. Chawla, “Heterogeneous graph neural network,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2019, pp. 793–803.
[28]
J. Zhao, X. Wang, C. Shi, B. Hu, G. Song, and Y. Ye, “Heterogeneous graph structure learning for graph neural networks,” in 31st AAAI Conf. Artif. Intell., 2021, pp. 4697–4705.
[29]
H. Ji, X. Wang, C. Shi, B. Wang, and P. Yu, “Heterogeneous graph propagation network,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 1, pp. 521–532, Jan. 2023.
[30]
W. Hu et al., “Open graph benchmark: Datasets for machine learning on graphs,” in Proc. Annu. Conf. Neural Inf. Process. Syst., 2020, pp. 22118–22133.
[31]
L. Yu, L. Sun, B. Du, C. Liu, W. Lv, and H. Xiong, “Heterogeneous graph representation learning with relation awareness,” 2021,.
[32]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Proc. Adv. Neural Inf. Process. Syst., 2013, pp. 3111–3119.
[33]
K. Wang, Z. Shen, C. Huang, C. Wu, Y. Dong, and A. Kanakia, “Microsoft academic graph: When experts are not enough,” Quant. Sci. Stud., vol. 1, no. 1, pp. 396–413, 2020.
[34]
Y. Dong, N. V. Chawla, and A. Swami, “metapath2vec: Scalable representation learning for heterogeneous networks,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2017, 2017, pp. 135–144.
[35]
F. Zhang et al., “OAG: Toward linking large-scale heterogeneous entity graphs,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2019, pp. 2585–2595.
[36]
Z. Yang, Z. Dai, Y. Yang, J. G. Carbonell, R. Salakhutdinov, and Q. V. Le, “Xlnet: Generalized autoregressive pretraining for language understanding,” in Proc. Annu. Conf. Neural Inf. Process. Syst., 2019, pp. 5754–5764.
[37]
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014,.
[38]
I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” in Proc. 5th Int. Conf. Learn. Representations, 2017.
[39]
N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, 2014.
[40]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770–778.
[41]
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. 32nd Int. Conf. Mach. Learn., 2015, pp. 448–456.
[42]
A. Paszke et al., “PyTorch: An imperative style, high-performance deep learning library,” in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 8024–8035.
[43]
M. Wang et al., “Deep graph library: Towards efficient and scalable deep learning on graphs,” 2019,.
[44]
G. Li, C. Xiong, A. Thabet, and B. Ghanem, “DeeperGCN: All you need to train deeper GCNs,” 2020,.
[45]
J. Zhang, X. Shi, J. Xie, H. Ma, I. King, and D. Yeung, “GAAN: Gated attention networks for learning on large and spatiotemporal graphs,” in Proc. 34th Conf. Uncertainty Artif. Intell., 2018, pp. 339–349.
[46]
M. Liu, H. Gao, and S. Ji, “Towards deeper graph neural networks,” in Proc. 26th ACM SIGKDD Conf. Knowl. Discov. Data Mining, 2020, 2020, pp. 338–348.
[47]
K. Xu, C. Li, Y. Tian, T. Sonobe, K.-I. Kawarabayashi, and S. Jegelka, “Representation learning on graphs with jumping knowledge networks,” in Proc. Int. Conf. Mach. Learn., 2018, pp. 5453–5462.
[48]
M. Chen, Z. Wei, Z. Huang, B. Ding, and Y. Li, “Simple and deep graph convolutional networks,” in Proc. Int. Conf. Mach. Learn., 2020, pp. 1725–1735.
[49]
Q. Huang, H. He, A. Singh, S. Lim, and A. R. Benson, “Combining label propagation and simple models out-performs graph neural networks,” in Proc. 9th Int. Conf. Learn. Representations, 2021.
[50]
E. Rossi, F. Frasca, B. Chamberlain, D. Eynard, M. M. Bronstein, and F. Monti, “SIGN: Scalable inception graph neural networks,” 2020,.
[51]
M. S. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in Proc. Semantic Web 15th Int. Conf., 2018, pp. 593–607.
[52]
Z. Hu, Y. Dong, K. Wang, and Y. Sun, “Heterogeneous graph transformer,” in Proc. Web Conf., 2020, pp. 2704–2710.
[53]
X. Wu, M. Jiang, and G. Liu, “R-GSN: The relation-based graph similar network for heterogeneous graph,” 2021,.
[54]
L. Yu, L. Sun, B. Du, C. Liu, W. Lv, and H. Xiong, “Hybrid micro/macro level convolution for heterogeneous graph learning,” 2020,.
[55]
S. Zhu, C. Zhou, S. Pan, X. Zhu, and B. Wang, “Relation structure-aware heterogeneous graph neural network,” in Proc. IEEE Int. Conf. Data Mining, 2019, pp. 1534–1539.
[56]
H. Hong, H. Guo, Y. Lin, X. Yang, Z. Li, and J. Ye, “An attention-based graph neural network for heterogeneous structural learning,” in Proc. 34th AAAI Conf. Artif. Intell., 2020, pp. 4132–4139.
[57]
L. v. d. Maaten and G. Hinton, “Visualizing data using T-SNE,” J. Mach. Learn. Res., vol. 9, no. Nov, pp. 2579–2605, 2008.
Index Terms
- Label-Enhanced Graph Neural Network for Semi-Supervised Node Classification
Index terms have been assigned to the content through auto-classification.
Recommendations
Semi-supervised multi-label classification using incomplete label information
Highlights- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
AbstractClassifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...
Semantic guide for semi-supervised few-shot multi-label node classification
AbstractWe study a new research problem named semi-supervised few-shot multi-label node classification which has the following characteristics: 1) the extreme imbalance between the number of labeled and unlabeled nodes that are connected on ...
Semi-supervised Multi-label k-Nearest Neighbors Classification Algorithms
BRACIS '15: Proceedings of the 2015 Brazilian Conference on Intelligent Systems (BRACIS)Classification is one of the most traditional tasks in machine learning. In supervised learning for classification, the goal is to learn a classifier function using a completely labeled dataset. Semi-supervised learning modifies the learning algorithm ...
Comments
Information & Contributors
Information
Published In
1041-4347 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 23 December 2022
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in