research-article

Public Access

A Unified View on Graph Neural Networks as Graph Signal Denoising

Authors:

Neil ShahAuthors Info & Claims

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

Pages 1202 - 1211

https://doi.org/10.1145/3459637.3482225

Published: 30 October 2021 Publication History

Abstract

Graph Neural Networks (GNNs) have risen to prominence in learning representations for graph structured data. A single GNN layer typically consists of a feature transformation and a feature aggregation operation. The former normally uses feed-forward networks to transform features, while the latter aggregates the transformed features over the graph. Numerous recent works have proposed GNN models with different designs in the aggregation operation. In this work, we establish mathematically that the aggregation processes in a group of representative GNN models including GCN, GAT, PPNP, and APPNP can be regarded as (approximately) solving a graph denoising problem with a smoothness assumption. Such a unified view across GNNs not only provides a new perspective to understand a variety of aggregation operations but also enables us to develop a unified graph neural network framework UGNN. To demonstrate its promising potential, we instantiate a novel GNN model, ADA-UGNN, derived from UGNN, to handle graphs with adaptive smoothness across nodes. Comprehensive experiments show the effectiveness of ADA-UGNN.

References

[1]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013).

[2]

Siheng Chen, Yonina C Eldar, and Lingxiao Zhao. 2020. Graph Unrolling Networks: Interpretable Neural Networks for Graph Signal Denoising. arXiv preprint arXiv:2006.01301 (2020).

[3]

Siheng Chen, Aliaksei Sandryhaila, José MF Moura, and Jelena Kovacevic. 2014. Signal denoising on graphs via graph filtering. In 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, 872--876.

[4]

Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. 2020. Adaptive universal generalized pagerank graph neural network. arXiv preprint arXiv:2006.07988 (2020).

[5]

Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Velivc ković. 2020. Principal neighbourhood aggregation for graph nets. arXiv preprint arXiv:2004.05718 (2020).

[6]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems. 3844--3852.

[7]

Guoji Fu, Yifan Hou, Jian Zhang, Kaili Ma, Barakeel Fanseu Kamhoua, and James Cheng. 2020. Understanding graph neural networks from graph signal denoising perspectives. arXiv preprint arXiv:2006.04386 (2020).

[8]

Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD. 1416--1424.

Digital Library

[9]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212 (2017).

[10]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024--1034.

[11]

Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163 (2015).

[12]

Xiao Huang, Jundong Li, and Xia Hu. 2017. Label informed attributed network embedding. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. 731--739.

Digital Library

[13]

Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. 2020. Graph Structure Learning for Robust Graph Neural Networks. arXiv preprint arXiv:2005.10203 (2020).

[14]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[15]

Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997 (2018).

[16]

Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. 2021. Is Homophily a Necessity for Graph Neural Networks? arXiv preprint arXiv:2106.06134 (2021).

[17]

Yao Ma and Jiliang Tang. 2021. Deep learning on graphs. Cambridge University Press.

[18]

Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. 2017. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE Conference on CVPR. 5115--5124.

[19]

Hoang NT and Takanori Maehara. 2019. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550 (2019).

[20]

Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287 (2020).

[21]

Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification. In International Conference on Learning Representations.

[22]

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine, Vol. 29, 3 (2008), 93--93.

[23]

Neil Shah. 2020. Scale-Free, Attributed and Class-Assortative Graph Generation to Facilitate Introspection of Graph Neural Networks. KDD Mining and Learning with Graphs (2020).

[24]

Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).

[25]

David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. 2013. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine, Vol. 30, 3 (2013), 83--98.

[26]

Ryan J Tibshirani et al. 2014. Adaptive piecewise polynomial estimation via trend filtering. The Annals of Statistics, Vol. 42, 1 (2014), 285--323.

[27]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[28]

Yu-Xiang Wang, James Sharpnack, Alexander J Smola, and Ryan J Tibshirani. 2016. Trend filtering on graphs. The Journal of Machine Learning Research, Vol. 17, 1 (2016), 3651--3691.

Digital Library

[29]

Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. 2019b. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610 (2019).

[30]

Jun Wu, Jingrui He, and Jiejun Xu. 2019a. Net: Degree-specific graph neural networks for node and graph classification. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 406--415.

Digital Library

[31]

Zhilin Yang, William Cohen, and Ruslan Salakhudinov. 2016. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning. PMLR, 40--48.

[32]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018a. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD. 974--983.

Digital Library

[33]

Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. 2018b. Hierarchical graph representation learning with differentiable pooling. In Advances in neural information processing systems. 4800--4810.

[34]

Lingxiao Zhao and Leman Akoglu. 2019. Pairnorm: Tackling oversmoothing in gnns. arXiv preprint arXiv:1909.12223 (2019).

[35]

Tong Zhao, Yozen Liu, Leonardo Neves, Oliver Woodford, Meng Jiang, and Neil Shah. 2020. Data Augmentation for Graph Neural Networks. arXiv preprint arXiv:2006.06830 (2020).

[36]

Jiong Zhu, Ryan A Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K Ahmed, and Danai Koutra. 2020a. Graph neural networks with heterophily. arXiv preprint arXiv:2009.13566 (2020).

[37]

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. 2020b. Beyond homophily in graph neural networks: Current limitations and effective designs. arXiv preprint arXiv:2006.11468 (2020).

[38]

Meiqi Zhu, Xiao Wang, Chuan Shi, Houye Ji, and Peng Cui. 2021. Interpreting and Unifying Graph Neural Networks with An Optimization Framework. arXiv preprint arXiv:2101.11859 (2021).

[39]

Daniel Zügner and Stephan Günnemann. 2019. Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv:1902.08412 (2019).

Cited By

Yan WLi ZQi Y(2024)Robust Regularization Design of Graph Neural Networks Against Adversarial Attacks Based on Lyapunov TheoryChinese Journal of Electronics10.23919/cje.2022.00.34233:3(732-741)Online publication date: May-2024
https://doi.org/10.23919/cje.2022.00.342
Zhang AHuang JLi PZhang K(2024)Building Shortcuts between Distant Nodes with Biaffine Mapping for Graph Convolutional NetworksACM Transactions on Knowledge Discovery from Data10.1145/365011318:6(1-21)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3650113
Kuo CChen BPeng WHung CSu H(2024)Correlation-aware Graph Data Augmentation with Implicit and Explicit NeighborsACM Transactions on Knowledge Discovery from Data10.1145/363805718:5(1-23)Online publication date: 25-Jan-2024
https://dl.acm.org/doi/10.1145/3638057
Show More Cited By

Index Terms

A Unified View on Graph Neural Networks as Graph Signal Denoising
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks

Recommendations

Toward the analysis of graph neural networks
ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results

Graph Neural Networks (GNNs) have recently emerged as an effective framework for representing and analyzing graph-structured data. GNNs have been applied to many real-world problems such as knowledge graph analysis, social networks recommendation, and ...
Graph Mining with Graph Neural Networks
WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Graphs are ubiquitous data structures in various fields, such as social media, transportation, linguistics and chemistry. To solve downstream graph-related tasks, it is of great significance to learn effective representations for graphs. My research ...
Streaming Graph Neural Networks
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Graphs are used to model pairwise relations between entities in many real-world scenarios such as social networks. Graph Neural Networks(GNNs) have shown their superior ability in learning representations for graph structured data, which leads to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

October 2021

4966 pages

ISBN:9781450384469

DOI:10.1145/3459637

General Chairs:
Gianluca Demartini
The University of Queensland, Australia
,
Guido Zuccon
The University of Queensland, Australia
,
Program Chairs:
J. Shane Culpepper
RMIT University, Australia
,
Zi Huang
The University of Queensland, Australia
,
Hanghang Tong
University of Illinois at Urbana-Champaign, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CIKM '21

Sponsor:

CIKM '21: The 30th ACM International Conference on Information and Knowledge Management

November 1 - 5, 2021

Queensland, Virtual Event, Australia

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
1,811
Total Downloads

Downloads (Last 12 months)791
Downloads (Last 6 weeks)55

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yan WLi ZQi Y(2024)Robust Regularization Design of Graph Neural Networks Against Adversarial Attacks Based on Lyapunov TheoryChinese Journal of Electronics10.23919/cje.2022.00.34233:3(732-741)Online publication date: May-2024
https://doi.org/10.23919/cje.2022.00.342
Zhang AHuang JLi PZhang K(2024)Building Shortcuts between Distant Nodes with Biaffine Mapping for Graph Convolutional NetworksACM Transactions on Knowledge Discovery from Data10.1145/365011318:6(1-21)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3650113
Kuo CChen BPeng WHung CSu H(2024)Correlation-aware Graph Data Augmentation with Implicit and Explicit NeighborsACM Transactions on Knowledge Discovery from Data10.1145/363805718:5(1-23)Online publication date: 25-Jan-2024
https://dl.acm.org/doi/10.1145/3638057
Khoshraftar SAn A(2024)A Survey on Graph Representation Learning MethodsACM Transactions on Intelligent Systems and Technology10.1145/363351815:1(1-55)Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1145/3633518
Wang LFan WLi JMa YLi QChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Fast Graph Condensation with Structure-based Neural Tangent KernelProceedings of the ACM on Web Conference 202410.1145/3589334.3645694(4439-4448)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645694
Wang BChen JLi CZhou SShi QGao YFeng YChen CWang CChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Distributionally Robust Graph-based Recommendation SystemProceedings of the ACM on Web Conference 202410.1145/3589334.3645598(3777-3788)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645598
Habib BIsufi Evan Breda WJongepier ACremer J(2024)Deep Statistical Solver for Distribution System State EstimationIEEE Transactions on Power Systems10.1109/TPWRS.2023.329035839:2(4039-4050)Online publication date: Mar-2024
https://doi.org/10.1109/TPWRS.2023.3290358
Xu CWang JZhang W(2024)StableGCN: Decoupling and Reconciling Information Propagation for Collaborative FilteringIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.332345836:6(2659-2670)Online publication date: Jun-2024
https://doi.org/10.1109/TKDE.2023.3323458
Yang JWang CYang LJiang YCao A(2024)Adaptive Feature Learning for Unbiased Scene Graph GenerationIEEE Transactions on Image Processing10.1109/TIP.2024.337464433(2252-2265)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3374644
Wei JWang YGao XHe RSun Z(2024)Multi-Faceted Knowledge-Driven Graph Neural Network for Iris SegmentationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.340750819(6015-6027)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3407508
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents