Hardware Efficiency Stochastic Computing based on Hybrid Spatial Coding
Article No.: 23, Pages 1 - 6
Abstract
As the era of silicon-based microchips advances by Moores Law approach to physical limits, new computational paradigms are proposed for future systems, i.e., stochastic computation. However, the current stochastic computing faces the challenge of high latency and low accuracy. In this work, we propose spatial coding based on the hybrid stochastic computation (SHSC) method, which is a stochastic-binary hybrid domain computation. Instead of sequential bits computing, the proposed SHSC expands stochastic bits in the spatial dimension. To balance the accuracy and complexity, the multiplication is divided into high and low precision parts, where the high precision parts are performed in the binary domain, and low precision parts are performed in the stochastic domain. A low-cost error compensation circuit is proposed to further improve the computation accuracy. According to the implementation outcomes, the proposed method exhibits a 28% hardware efficiency improvement with the same inference accuracy as traditional neural network applications.
References
[1]
A. Zhou, A. Yao, Y. Guo, L. Xu, and Y. Chen. Incremental network quantization: Towards lossless cnns with low-precision weights. 2017.
[2]
M. Courbariaux, Y. Bengio, and J. P. David. Binaryconnect: Training deep neural networks with binary weights during propagations. In International Conference on Neural Information Processing Systems, 2015.
[3]
Valentino Peluso and Andrea Calimera. Energy-accuracy scalable deep convolutional neural networks: A pareto analysis. In Nicola Bombieri, Graziano Pravadelli, Masahiro Fujita, Todd Austin, and Ricardo Reis, editors, VLSI-SoC: Design and Engineering of Electronics Systems Based on New Computing Paradigms, pages 107--127, Cham, 2019. Springer International Publishing.
[4]
C. Y. Chen, J. Choi, K. Gopalakrishnan, V. Srinivasan, and S. Venkataramani. Exploiting approximate computing for deep learning acceleration. In Design, Automation and Test in Europe Conference and Exhibition, pages 821--826, 2018.
[5]
C. Cheng, P. J. Tiw, Y. Cai, X. Yan, Y. Yang, and R. Huang. In-memory computing with emerging nonvolatile memory devices. SCIENCE CHINA Information Sciences, 64(12):221402, 2021.
[6]
X. Qian. Graph processing and machine learning architectures with emerging memory technologies:a survey. SCIENTIA SINICA Informationis, 64(6):25, 2021.
[7]
Tianyun Zhang, Shaokai Ye, Kaiqi Zhang, Jian Tang, Wujie Wen, Makan Fardad, and Yanzhi Wang. A systematic dnn weight pruning framework using alternating direction method of multipliers. In ECCV (8), pages 191--207, 2018.
[8]
L. Guo, D. Chen, and K. Jia. Knowledge transferred adaptive filter pruning for cnn compression and acceleration. Science China Information Sciences, 65(12):1--2, 2022.
[9]
A. Carpegna, A. Savino, and S. D. Carlo. Spiker: an fpga-optimized hardware acceleration for spiking neural networks. 2022.
[10]
Jienan Chen, Siyu Chen, Qi Wang, Bin Cao, Gang Feng, and Jianhao Hu. iraf: A deep reinforcement learning approach for collaborative mobile edge computing iot networks. IEEE Internet of Things Journal, 6(4):7011--7024, 2019.
[11]
V. Sze, Y. H. Chen, T. J. Yang, and J. S. Emer. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE, 105(12), 2017.
[12]
Gijs A. G. M. Hendriks, Gert Weijers, Chuan Chen, Madeleine Hertel, Chi-Yin Lee, Peter M. Dueppenbecker, Marcus Radicke, Andy Milkowski, Hendrik H. G. Hansen, and Chris L. de Korte. Comprehensive comparison of image quality aspects between conventional and plane-wave imaging methods on a commercial scanner. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 69(6):2039--2049, 2022.
[13]
R. Andri, L. Cavigelli, D. Rossi, and L. Benini. Yodann: An architecture for ultra-low power binary-weight cnn acceleration. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, PP(99):1--1, 2017.
[14]
H. Song, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In ICLR, 2016.
[15]
J. Chen, S. Li, J. Tao, S. Fu, and G. E. Sobelman. Wireless beam modulation: An energy- and spectrum-efficient communication technology for future massive iot systems. IEEE Wireless Communications, 27(5):60--66, 2020.
[16]
B. Reagen, P. Whatmough, R. Adolf, S. Rama, H. Lee, S. Lee, J. Hernandezlobato, G. Wei, and D. Brooks. Minerva: Enabling low-power, highly-accurate deep neural network accelerators. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), pages 267--278, 2016.
[17]
J. Chen, S. Chen, Y. Qi, and S. Fu. Intelligent massive mimo antenna selection using monte carlo tree search. IEEE Transactions on Signal Processing, PP(99):1--1, 2019.
[18]
Z. Xia, J. Chen, Q. Huang, J. Luo, and J. Hu. Neural synaptic plasticity-inspired computing: A high computing efficient deep convolutional neural network accelerator. Circuits and Systems I: Regular Papers, IEEE Transactions on, PP(99):1--13, 2020.
[19]
Y. Zhang, R. Wang, X. Zhang, Y. Wang, and R. Huang. Parallel hybrid stochastic-binary-based neural network accelerators. Circuits and Systems II: Express Briefs, IEEE Transactions on, PP(99):1--1, 2020.
[20]
Hyeonuk Sim and Jongeun Lee. A new stochastic computing multiplier with application to deep convolutional neural networks. In 2017 54th ACM/EDAC/IEEE Design Automation Conference (DAC), pages 1--6, 2017.
[21]
Xia Zihan, Wan Rui, Chen Jienan, and Wang Runsheng. Reconfigurable spatial parallel stochastic computing for accelerating sparse convolutional neural networks. SCIENCE CHINA Information Sciences. Received 16 November 2022; revised 20 November 2022; accepted 27 November 2022
Index Terms
- Hardware Efficiency Stochastic Computing based on Hybrid Spatial Coding
Recommendations
Multi-view video coding based on high efficiency video coding
PSIVT'11: Proceedings of the 5th Pacific Rim conference on Advances in Image and Video Technology - Volume Part IIMultiview video coding is one of the key techniques to realize the 3D video system. MPEG started a standardization activity on 3DVC (3D video coding) in 2007. 3DVC is based on multiview video coding. MPEG finalized the standard for multiview video ...
HDTV coding using hybrid MRVQ/DCT
The authors develop a technique to compress HDTV images to below 135 Mb/s at transparent quality, i.e., visually indistinguishable from the original. This is at least equal to the contribution quality. The proposed scheme consists of a lossy and a ...
A low energy intra prediction hardware for high efficiency video coding
Intra prediction algorithm in the recently developed high efficiency video coding (HEVC) standard has very high computational complexity. Therefore, in this paper, we propose pixel equality and pixel similarity based techniques for reducing amount of ...
Comments
Information & Contributors
Information
Published In
December 2022
140 pages
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Sichuan Science and Technology Program
Conference
NANOARCH '22
Sponsor:
NANOARCH '22: 17th ACM International Symposium on Nanoscale Architectures
December 7 - 9, 2022
OR, Virtual, USA
Acceptance Rates
NANOARCH '22 Paper Acceptance Rate 25 of 31 submissions, 81%;
Overall Acceptance Rate 55 of 87 submissions, 63%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 42Total Downloads
- Downloads (Last 12 months)24
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in