Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3453688.3461741acmconferencesArticle/Chapter ViewAbstractPublication PagesglsvlsiConference Proceedingsconference-collections
research-article

Co-Exploration of Graph Neural Network and Network-on-Chip Design Using AutoML

Published: 22 June 2021 Publication History

Abstract

Recently, Graph Neural Networks (GNNs) have exhibited high efficiency in several graph-based machine learning tasks. Compared with the neural networks for computer vision or speech tasks (e.g., Convolutional Neural Networks), GNNs have much higher requirements on communication due to the complicated graph structures; however, when applying GNNs for real-world applications, say in recommender systems (e.g. Uber Eats), it commonly has the real-time requirements. To deal with the tradeoff between the complicated architecture and the high-demand timing performance, both GNN architecture and hardware accelerator need to be optimized. Network-on-Chip (NoC), derived for efficiently managing the high-volume of communications, naturally becomes one of the top candidates to accelerate GNNs. However, there is a missing link between the optimize of GNN architecture and the NoC design.
In this work, we present an AutoML-based framework GN-NAS, aiming at searching for the optimum GNN architecture, which can be suitable for the NoC accelerator. We devise a robust reinforcement learning based controller to validate the retained best GNN architectures, coupled with a parameter sharing approach, namely ParamShare, to improve search efficiency. Experimental results on four graph-based benchmark datasets, Cora, Citeseer, Pubmed and Protein-Protein Interaction show that the GNN architectures obtained by our framework outperform that of the state-of-the-art and baseline models, whilst reducing model size which makes them easy to deploy onto the NoC platform.

Supplemental Material

MP4 File
This work entails the design of an Auto-ML based framework called GN-NAS to search for the optimal GNN architectures within the parameter search space such that they are also implementable on the NoC platform to analyze HW specifications. The designed GN-NAS framework consists of 3 parts: Parameter search space, GNAS evaluator and NoC platform. The designed GN-NAS controller has 3 phases: GNN architecture exploration, RL training and Parameter modification. We have also proposed a parameter sharing strategy called ?ParamShare? which ensures that already trained weights in the parent architecture are transferred to the child architecture. With the datasets, we consider the citation graph datasets (Cora, Citeseer and Pubmed) and PPI which we evaluate under transductive and inductive settings respectively. Based on our experimental results, GN-NAS models achieves state-of-the-art performance for both transductive and inductive learning. We also observe that GNN models with ParamShare have smaller model sizes as compared to models without ParamShare. This shows that our designed GN-NAS framework can achieve a better trade-of between performance and model sizes.

References

[1]
Sergi Abadal, Akshay Jain, Robert Guirado, Jorge Lopez-Alonso, and Eduard Alarcon. 2020. Computing Graph Neural Networks: A Survey from Algorithms to Accelerators. 1--12. arXiv:2010.00130 [cs.LG]
[2]
Joan Bruna, W. Zaremba, Arthur D. Szlam, and Y. LeCun. 2014. Spectral Networks and Locally Connected Networks on Graphs. CoRR abs/1312.6203 (2014).
[3]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2017. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. arXiv:1606.09375 [cs.LG]
[4]
Radwa Elshawi, Mohamed Maher, and Sherif Sakr. 2019. Automated Machine Learning: State-of-The-Art and Open Challenges. arXiv:1906.02287 [cs.LG]
[5]
Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-Scale Learnable Graph Convolutional Networks. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining (2018).
[6]
M. Gori, G. Monfardini, and F. Scarselli. 2005. A new model for learning in graph domains. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., Vol. 2. 729--734 vol. 2. https://doi.org/10.1109/IJCNN.2005.1555942
[7]
Hantao Guo, Rui Yan, Yansong Feng, Xuesong Gao, and Zhanxing Zhu. 2020. Simplifying Graph Attention Networks with Source-Target Separation. In 24th European Conference on Artificial Intelligence - ECAI 2020). ECAI, 1--7.
[8]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. CoRR abs/1706.02216 (2017). arXiv:1706.02216
[9]
Cong Hao, Yao Chen, Xinheng Liu, Atif Sarwari, Daryl Sew, Ashutosh Dhar, Bryan Wu, Dongdong Fu, Jinjun Xiong, Wen mei Hwu, Junli Gu, and Deming Chen. 2019. NAIS: Neural Architecture and Implementation Search and its Applications in Autonomous Driving. arXiv:1911.07446
[10]
Jiechuan Jiang, Chen Dun, Tiejun Huang, and Zongqing Lu. 2020. Graph Convolutional Reinforcement Learning. In International Conference on Learning Representations. 1--13. arXiv:1810.09202 [cs.LG]
[11]
Weiwen Jiang, Edwin Hsing-Mean Sha, Qingfeng Zhuge, Lei Yang, Xianzhang Chen, and Jingtong Hu. 2018. Heterogeneous fpga-based cost-optimal design for timing-constrained cnns. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37, 11 (2018), 2542--2554.
[12]
Weiwen Jiang, Lei Yang, Sakyasingha Dasgupta, Jingtong Hu, and Yiyu Shi. 2020. Standing on the shoulders of giants: Hardware and neural architecture co-search with hot start. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 11 (2020), 4154--4165.
[13]
Weiwen Jiang, Lei Yang, Edwin Hsing-Mean Sha, Qingfeng Zhuge, Shouzhen Gu, Sakyasingha Dasgupta, Yiyu Shi, and Jingtong Hu. 2020. Hardware/software co-exploration of Neural Architectures. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 12 (2020), 4805--4815.
[14]
Weiwen Jiang, Xinyi Zhang, Edwin H-M Sha, Lei Yang, Qingfeng Zhuge, Yiyu Shi, and Jingtong Hu. 2019. Accuracy vs. e!ciency: Achieving Both Through FPGA-Implementation Aware Neural Architecture Search. In Proceedings of the 56th Annual Design Automation Conference 2019. 1--6.
[15]
Weiwen Jiang, Xinyi Zhang, Edwin H-M Sha, Qingfeng Zhuge, Lei Yang, Yiyu Shi, and Jingtong Hu. 2019. Xfer: A novel design to achieve super-linear performance on multiple fpgas for real-time ai. In Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 305--305.
[16]
Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907 (2016). arXiv:1609.02907
[17]
Tahir Maqsood, Sabeen Ali, Saif U.R. Malik, and Sajjad A. Madani. 2015. Dynamic Task Mapping for Network-on-Chip Based Systems. J. Syst. Archit. 61, 7 (2015), 293--306. https://doi.org/10.1016/j.sysarc.2015.06.001
[18]
Matheus Nunes and Gisele L. Pappa. 2020. Intelligent Systems. Lecture Notes in Computer Science (2020). https://doi.org/10.1007/978-3-030-61377-8
[19]
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. 2009. The Graph Neural Network Model. IEEE Transactions on Neural Networks 20, 1 (2009), 61--80. https://doi.org/10.1109/TNN.2008.2005605
[20]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. arXiv:1710.10903 [stat.ML]
[21]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How Powerful are Graph Neural Networks" CoRR abs/1810.00826 (2018). arXiv:1810.00826
[22]
M. Yan, L. Deng, X. Hu, L. Liang, Y. Feng, X. Ye, Z. Zhang, D. Fan, and Y. Xie. 2020. HyGCN: A GCN Accelerator with Hybrid Architecture. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). 15--29. https://doi.org/10.1109/HPCA47549.2020.00012
[23]
Lei Yang, Weiwen Jiang, Weichen Liu, Edwin H. M. Sha, Yiyu Shi, and Jingtong Hu. 2020. Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence. In 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC). 85--90. https://doi.org/10.1109/ASP-DAC47756.2020. 9045595
[24]
Lei Yang, Weichen Liu, Weiwen Jiang, Mengquan Li, Juan Yi, and Edwin Hsing-Mean Sha. 2016. Application mapping and scheduling for network-on-chip-based multiprocessor system-on-chip with fine-grain communication optimization. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 24, 10 (2016), 3027--3040.
[25]
Lei Yang, Zheyu Yan, Meng Li, Hyoukjun Kwon, Liangzhen Lai, Tushar Krishna, Vikas Chandra, Weiwen Jiang, and Yiyu Shi. 2020. Co-exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks. In 2020 57th ACM/IEEE Design Automation Conference (DAC). IEEE, 1--6.
[26]
Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. 2016. Revisiting Semi-Supervised Learning with Graph Embeddings. arXiv:1603.08861 [cs.LG]
[27]
Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. 2018. GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs. CoRR abs/1803.07294 (2018). arXiv:1803.07294
[28]
Xinyi Zhang, Weiwen Jiang, Yiyu Shi, and Jingtong Hu. 2019. When neural architecture search meets Hardware Implementation: From Hardware Awareness to Co-design. In 2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). IEEE, 25--30.
[29]
Tong Zhao, Chuchen Deng, K. Yu, Tianwen Jiang, Daheng Wang, and Meng Jiang. 2020. GNN-based Graph Anomaly Detection with Graph Anomaly Loss.
[30]
Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2019. Robust Graph Convolutional Networks Against Adversarial Attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining. Association for Computing Machinery, 1399--1407

Cited By

View all
  • (2024)TSTL-GNN: Graph-Based Two-Stage Transfer Learning for Timing Engineering Change Order Analysis AccelerationElectronics10.3390/electronics1315289713:15(2897)Online publication date: 23-Jul-2024
  • (2024)Sparsifying Graph Neural Networks with Compressive SensingProceedings of the Great Lakes Symposium on VLSI 202410.1145/3649476.3658780(315-318)Online publication date: 12-Jun-2024
  • (2024)GraphGANFed: A Federated Generative Framework for Graph-Structured Molecules Towards Efficient Drug DiscoveryIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2024.334999021:2(240-253)Online publication date: 5-Jan-2024
  • Show More Cited By

Index Terms

  1. Co-Exploration of Graph Neural Network and Network-on-Chip Design Using AutoML

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      GLSVLSI '21: Proceedings of the 2021 Great Lakes Symposium on VLSI
      June 2021
      504 pages
      ISBN:9781450383936
      DOI:10.1145/3453688
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 June 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. automl
      2. graph neural network
      3. network-on-chip

      Qualifiers

      • Research-article

      Data Availability

      This work entails the design of an Auto-ML based framework called GN-NAS to search for the optimal GNN architectures within the parameter search space such that they are also implementable on the NoC platform to analyze HW specifications. The designed GN-NAS framework consists of 3 parts: Parameter search space, GNAS evaluator and NoC platform. The designed GN-NAS controller has 3 phases: GNN architecture exploration, RL training and Parameter modification. We have also proposed a parameter sharing strategy called ?ParamShare? which ensures that already trained weights in the parent architecture are transferred to the child architecture. With the datasets, we consider the citation graph datasets (Cora, Citeseer and Pubmed) and PPI which we evaluate under transductive and inductive settings respectively. Based on our experimental results, GN-NAS models achieves state-of-the-art performance for both transductive and inductive learning. We also observe that GNN models with ParamShare have smaller model sizes as compared to models without ParamShare. This shows that our designed GN-NAS framework can achieve a better trade-of between performance and model sizes. https://dl.acm.org/doi/10.1145/3453688.3461741#GLSVLSI21-vlsi14s.mp4

      Conference

      GLSVLSI '21
      Sponsor:
      GLSVLSI '21: Great Lakes Symposium on VLSI 2021
      June 22 - 25, 2021
      Virtual Event, USA

      Acceptance Rates

      Overall Acceptance Rate 312 of 1,156 submissions, 27%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)32
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 05 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)TSTL-GNN: Graph-Based Two-Stage Transfer Learning for Timing Engineering Change Order Analysis AccelerationElectronics10.3390/electronics1315289713:15(2897)Online publication date: 23-Jul-2024
      • (2024)Sparsifying Graph Neural Networks with Compressive SensingProceedings of the Great Lakes Symposium on VLSI 202410.1145/3649476.3658780(315-318)Online publication date: 12-Jun-2024
      • (2024)GraphGANFed: A Federated Generative Framework for Graph-Structured Molecules Towards Efficient Drug DiscoveryIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2024.334999021:2(240-253)Online publication date: 5-Jan-2024
      • (2022)Seismic Waveform Inversion Capability on Resource-Constrained Edge DevicesJournal of Imaging10.3390/jimaging81203128:12(312)Online publication date: 22-Nov-2022
      • (2022)Towards Sparsification of Graph Neural Networks2022 IEEE 40th International Conference on Computer Design (ICCD)10.1109/ICCD56317.2022.00048(272-279)Online publication date: Oct-2022

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media