research-article

DANCE: Differentiable Accelerator/Network Co-Exploration

Authors:

Jinho LeeAuthors Info & Claims

2021 58th ACM/IEEE Design Automation Conference (DAC)

Pages 337 - 342

https://doi.org/10.1109/DAC18074.2021.9586121

Published: 05 December 2021 Publication History

Abstract

This work presents DANCE, a differentiable approach towards the co-exploration of hardware accelerator and network architecture design. At the heart of DANCE is a differentiable evaluator network. By modeling the hardware evaluation software with a neural network, the relation between the accelerator design and the hardware metrics becomes differentiable, allowing the search to be performed with backpropagation. Compared to the naive existing approaches, our method performs co-exploration in a significantly shorter time, while achieving superior accuracy and hardware cost metrics.

References

[1]

David Silver et al. “Mastering the game of go without human knowledge”. In: Nature 550.7676 (2017), pp. 354–359.

[2]

Hanxiao Liu, Karen Simonyan, and Yiming Yang. “DARTS: Differentiable Architecture Search”. In: ICLR. 2018.

[3]

Mingxing Tan et al. “Mnasnet: Platform-aware neural architecture search for mobile”. In: CVPR. 2019, pp. 2820–2828.

[4]

Han Cai, Ligeng Zhu, and Song Han. “ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware”. In: ICLR. 2018.

[5]

Tianshi Chen et al. “Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning”. In: ASPLOS. 2014.

[6]

Yu-Hsin Chen et al. “Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks”. In: IEEE JSSC 52.1 (2016), pp. 127–138.

[7]

Norman P Jouppi et al. “In-datacenter performance analysis of a tensor processing unit”. In: ISCA. 2017.

[8]

Andrew G Howard et al. “Mobilenets: Efficient convolutional neural networks for mobile vision applications”. In: arXiv preprint arXiv:1704.04861 (2017).

[9]

Suyog Gupta and Berkin Akin. “Accelerator-aware Neural Network Design using AutoML”. In: arXiv preprint arXiv:2003.02838 (2020).

[10]

Weiwen Jiang et al. “Hardware/Software co-exploration of neural architectures”. In: IEEE TCAD (2020).

[11]

Cong Hao et al. “FPGA/DNN Co-Design: An Efficient Design Methodology for 1oT Intelligence on the Edge”. In: DAC. IEEE. 2019, pp. 1–6.

[12]

Qing Lu et al. “On neural architecture search for resource-constrained hardware platforms”. In: arXiv preprint arXiv:1911.00105 (2019).

[13]

Lei Yang et al. “Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks”. In: arXiv preprint arXiv:2002.04116 (2020).

[14]

Mohamed S Abdelfattah et al. “Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Accelerator”. In: arXiv preprint arXiv:2002.05022 (2020).

[15]

Angshuman Parashar et al. “Timeloop: A Systematic Approach to DNN Accelerator Evaluation”. In: ISPASS. 2019.

[16]

Hyoukjun Kwon et al. “Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach”. In: MICRO. 2019.

[17]

Yuhong Li et al. “EDD: Efficient Differentiable DNN Architecture and Implementation Co-Search for Embedded AI Solutions”. In: DAC. 2020.

[18]

Ananda Samajdar et al. “Scale-sim: Systolic cnn accelerator simulator”. In: arXiv preprint arXiv:1811.02883 (2018).

[19]

Esteban Real et al. “Regularized evolution for image classifier architecture search”. In: AAAI. 2019.

[20]

Bichen Wu et al. “Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search”. In: CVPR. 2019, pp. 10734–10742.

[21]

Yuhui Xu et al. “Latency-Aware Differentiable Neural Architecture Search”. In: arXiv preprint arXiv:2001.06392 (2020).

[22]

Maxim Berman et al. “AOWS: Adaptive and optimal network width search with latency constraints”. In: CVPR. 2020, pp. 11217–11226.

[23]

Han Cai et al. “Once-for-All: Train One Network and Specialize it for Efficient Deployment”. In: ICLR. 2019.

[24]

Zidong Du et al. “ShiDianNao: Shifting vision processing closer to the sensor”. In: ISCA. 2015.

[25]

Yannan Nellie Wu, Joel S. Emer, and Vivienne Sze. “Accelergy: An Architecture-Level Energy Estimation Methodology for Accelerator Designs”. In: ICCAD. 2019.

[26]

Yongan Zhang et al. “DNA: Differentiable Network-Accelerator Co-Search”. In: arXiv preprint arXiv:2010.14778 (2020).

[27]

Kaiming He et al. “Deep residual learning for image recognition”. In: CVPR. 2016.

[28]

Eric Jang, Shixiang Gu, and Ben Poole. “Categorical reparameterization with gumbel-softmax”. In: ICLR (2017).

[29]

Chuan Li. ResNet9: train to 94% CIFAR10 accuracy in 100 seconds with a single Turing GPU. 2019. URL: https://lambdalabs.com/blog/resnet9-train-to-94-cifar10-accuracy-in-100-seconds

Cited By

Liu HGalindo MXie HWong LShuai HLi YCheng W(2024)Lightweight Deep Learning for Resource-Constrained Environments: A SurveyACM Computing Surveys10.1145/365728256:10(1-42)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3657282
Dave SNowatzki TShrivastava AAamodt TSwift MJerger N(2023)Explainable-DSE: An Agile and Explainable Exploration of Efficient HW/SW Codesigns of Deep Learning Accelerators Using Bottleneck AnalysisProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 410.1145/3623278.3624772(87-107)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3623278.3624772
Bhattacharjee AMoitra APanda P(2023) XploreNAS: Explore Adversarially Robust and Hardware-efficient Neural Architectures for Non-ideal XbarsACM Transactions on Embedded Computing Systems10.1145/359304522:4(1-17)Online publication date: 24-Jul-2023
https://dl.acm.org/doi/10.1145/3593045
Show More Cited By

Index Terms

DANCE: Differentiable Accelerator/Network Co-Exploration

Index terms have been assigned to the content through auto-classification.

Recommendations

An FPGA-based accelerator platform implements for convolutional neural network
HP3C '19: Proceedings of the 3rd International Conference on High Performance Compilation, Computing and Communications

In recent years, convolutional neural network (CNN) has become widely universal in large number of applications including computer vision, natural language processing and automatic driving. However, the CNN-based methods are computational-intensive and ...
Configurable Network Protocol Accelerator (COPA)
Today's field-programmable gate arrays (FPGAs) offer not only programmable acceleration capabilities but also include advanced features that are on-par with a mainstream processor platform. However, rather than being deployed as autonomous acceleration ...
Enabling hard constraints in differentiable neural network and accelerator co-exploration
DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

2021 58th ACM/IEEE Design Automation Conference (DAC)

Dec 2021

1380 pages

Copyright © 2021.

Publisher

IEEE Press

Publication History

Published: 05 December 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu HGalindo MXie HWong LShuai HLi YCheng W(2024)Lightweight Deep Learning for Resource-Constrained Environments: A SurveyACM Computing Surveys10.1145/365728256:10(1-42)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3657282
Dave SNowatzki TShrivastava AAamodt TSwift MJerger N(2023)Explainable-DSE: An Agile and Explainable Exploration of Efficient HW/SW Codesigns of Deep Learning Accelerators Using Bottleneck AnalysisProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 410.1145/3623278.3624772(87-107)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3623278.3624772
Bhattacharjee AMoitra APanda P(2023) XploreNAS: Explore Adversarially Robust and Hardware-efficient Neural Architectures for Non-ideal XbarsACM Transactions on Embedded Computing Systems10.1145/359304522:4(1-17)Online publication date: 24-Jul-2023
https://dl.acm.org/doi/10.1145/3593045
Tuli SLi CSharma RJha N(2023)CODEBench: A Neural Architecture and Hardware Accelerator Co-Design FrameworkACM Transactions on Embedded Computing Systems10.1145/357579822:3(1-30)Online publication date: 20-Apr-2023
https://dl.acm.org/doi/10.1145/3575798
Mendis HKang CHsiu P(2021)Intermittent-Aware Neural Architecture SearchACM Transactions on Embedded Computing Systems10.1145/347699520:5s(1-27)Online publication date: 17-Sep-2021
https://dl.acm.org/doi/10.1145/3476995

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents