research-article

A Fast and Power Efficient Architecture to Parallelize LSTM based RNN for Cognitive Intelligence Applications

Authors:

Peng Ouyang,

Shouyi Yin,

Shaojun WeiAuthors Info & Claims

DAC '17: Proceedings of the 54th Annual Design Automation Conference 2017

Article No.: 63, Pages 1 - 6

https://doi.org/10.1145/3061639.3062187

Published: 18 June 2017 Publication History

Get Access

Abstract

Long Short-Term Memory (LSTM) based Recurrent Neural Networks (RNNs) are promising for cognitive intelligence applications like speech recognition, image caption and nature language processing, etc. However, the cascade dependent structure in RNN with huge amount of power inefficient operations like multiplication, memory accessing and nonlinear transformation, could not guarantee high computing speed and low power consumption. In this work, by exploiting semantic correlation, we propose a semantic correlation based data pre-fetch method to break the dependency and achieve parallel processing. Based on this method, a full parallel and pipeline architecture that tackles huge amount operations is designed. Experiments on benchmarks of image caption, speech recognition and language processing show that, this work improves computing speed by 5.1 times, 44.9 times and 1.53 times, respectively, and power efficiency by 1885.7 times, 4061.5 times and 127.5 times, respectively, when compared with state-of-the-art works.

References

[1]

A. X. M. Chang, M. Berin, and C. Eugenio. Recurrent neural networks hardware implementation on fpga. arXiv preprint arXiv:1511.05552, 2015.

Google Scholar

[2]

J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, and etc. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2625--2634, 2015.

Crossref

Google Scholar

[3]

A. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1764--1772, 2014.

Digital Library

Google Scholar

[4]

K. Hwang and W. Sung. Single stream parallelization of generalized lstm-like rnns on a gpu. In Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on, pages 1047--1051, 2015.

Crossref

Google Scholar

[5]

S. Li, C. Wu, H. Li, B. Li, Y. Wang, and Q. Qiu. Fpga acceleration of recurrent neural network based language model. In Field-Programmable Custom Computing Machines (FCCM), 2015 IEEE 23rd Annual International Symposium on, pages 111--118, 2015.

Digital Library

Google Scholar

[6]

Y. Miao, M. Gowayyed, and F. Metze. Eesen: End-to-end speech recognition using deep rnn models and wfst-based decoding. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pages 167--174, Dec 2015.

Crossref

Google Scholar

[7]

S. Shin, K. Hwang, and W. Sung. Fixed point performance analysis of recurrent neural networks. arXiv preprint arXiv:1512.01322, 2015.

Google Scholar

[8]

M. F. Stollenga, W. Byeon, M. Liwicki, and J. Schmidhuber. Parallel multi-dimensional lstm, with application to fast biomedical volumetric image segmentation. In Advances in Neural Information Processing Systems, pages 2980--2988, 2015.

Digital Library

Google Scholar

[9]

D. Subhasis and H. Song. Neuraltalk on embedded system and gpu-accelerated rn. In Technique report from standfor univerisyt, 2015.

Google Scholar

[10]

I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104--3112, 2014.

Digital Library

Google Scholar

Cited By

View all

Zhao XWang HBai MXu YDong SRao HMing W(2024)A Comprehensive Review of Methods for Hydrological Forecasting Based on Deep LearningWater10.3390/w1610140716:10(1407)Online publication date: 15-May-2024
https://doi.org/10.3390/w16101407
He WLiu TMing WLi ZDu JLi XGuo XSun P(2024)Progress in prediction of remaining useful life of hydrogen fuel cells based on deep learningRenewable and Sustainable Energy Reviews10.1016/j.rser.2023.114193192(114193)Online publication date: Mar-2024
https://doi.org/10.1016/j.rser.2023.114193
Li ZLu SZhang RZhao ZLiang RXue RLi WZhang FGao S(2023)VulHunter: Hunting Vulnerable Smart Contracts at EVM Bytecode-Level via Multiple Instance LearningIEEE Transactions on Software Engineering10.1109/TSE.2023.331720949:11(4886-4916)Online publication date: Nov-2023
https://doi.org/10.1109/TSE.2023.3317209
Show More Cited By

Recommendations

Fast Training of Deep LSTM Networks
Advances in Neural Networks – ISNN 2019
Abstract
Deep recurrent neural networks (RNN), such as LSTM, have many advantages over forward networks. However, the LSTM training method, such as backward propagation through time (BPTT), is really slow.
In this paper, by separating the LSTM cell into ...
Momentum-based wavelet and double wavelet neural networks for power system applications

In order to minimize the power loss and to control the voltage in the power systems, the proposed momentum-based wavelet neural network and proposed momentum-based double wavelet neural network are proposed in this paper. The training data are obtained ...
Text-independent speaker recognition using LSTM-RNN and speech enhancement
Abstract
Speaker recognition revolution has lead to the inclusion of speaker recognition modules in several commercial products. Most published algorithms for speaker recognition focus on text-dependent speaker recognition. In contrast, text-independent ...

Comments

Information & Contributors

Information

Published In

DAC '17: Proceedings of the 54th Annual Design Automation Conference 2017

June 2017

533 pages

ISBN:9781450349277

DOI:10.1145/3061639

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

China National High Technologies Research Program
China Major S&T Project

Conference

DAC '17

Sponsor:

EDAC
SIGDA

DAC '17: The 54th Annual Design Automation Conference 2017

June 18 - 22, 2017

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25

Sponsor:
sigda

62nd ACM/IEEE Design Automation Conference

June 22 - 26, 2025

San Francisco , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
558
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)1

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhao XWang HBai MXu YDong SRao HMing W(2024)A Comprehensive Review of Methods for Hydrological Forecasting Based on Deep LearningWater10.3390/w1610140716:10(1407)Online publication date: 15-May-2024
https://doi.org/10.3390/w16101407
He WLiu TMing WLi ZDu JLi XGuo XSun P(2024)Progress in prediction of remaining useful life of hydrogen fuel cells based on deep learningRenewable and Sustainable Energy Reviews10.1016/j.rser.2023.114193192(114193)Online publication date: Mar-2024
https://doi.org/10.1016/j.rser.2023.114193
Li ZLu SZhang RZhao ZLiang RXue RLi WZhang FGao S(2023)VulHunter: Hunting Vulnerable Smart Contracts at EVM Bytecode-Level via Multiple Instance LearningIEEE Transactions on Software Engineering10.1109/TSE.2023.331720949:11(4886-4916)Online publication date: Nov-2023
https://doi.org/10.1109/TSE.2023.3317209
Venkatachalam PLiu D(2023)On Hybrid Artificial Neural Networks and Variational Quantum Classifier for Network Intrusion Detection2023 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC)10.1109/CyberC58899.2023.00070(410-416)Online publication date: 2-Nov-2023
https://doi.org/10.1109/CyberC58899.2023.00070
He WLi ZLiu TLiu ZGuo XDu JLi XSun PMing W(2023)Research progress and application of deep learning in remaining useful life, state of health and battery thermal management of lithium batteriesJournal of Energy Storage10.1016/j.est.2023.10786870(107868)Online publication date: Oct-2023
https://doi.org/10.1016/j.est.2023.107868
Gao MChen HLiu D(2022)An ASIP for Neural Network Inference on Embedded Devices with 99% PE Utilization and 100% Memory Hidden under Low Silicon CostSensors10.3390/s2210384122:10(3841)Online publication date: 19-May-2022
https://doi.org/10.3390/s22103841
Wang HQiu DGe FYang Y(2022)Implementation of Bidirectional LSTM Accelerator Based on FPGA2022 IEEE 22nd International Conference on Communication Technology (ICCT)10.1109/ICCT56141.2022.10072756(1512-1516)Online publication date: 11-Nov-2022
https://doi.org/10.1109/ICCT56141.2022.10072756
Zuo JLv HZhou DXue QJin LZhou WYang DZhang C(2021)Deep learning based prognostic framework towards proton exchange membrane fuel cell for automotive applicationApplied Energy10.1016/j.apenergy.2020.115937281(115937)Online publication date: Jan-2021
https://doi.org/10.1016/j.apenergy.2020.115937
Lang RZhu SWang D(2021)Pitch contours curve frequency domain fitting with vocabulary matching based music generationMultimedia Tools and Applications10.1007/s11042-021-11049-xOnline publication date: 4-Jun-2021
https://doi.org/10.1007/s11042-021-11049-x
Zini JRizk YAwad M(2020)An Optimized Parallel Implementation of Non-Iteratively Trained Recurrent Neural NetworksJournal of Artificial Intelligence and Soft Computing Research10.2478/jaiscr-2021-000311:1(33-50)Online publication date: 3-Dec-2020
https://doi.org/10.2478/jaiscr-2021-0003
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Fast Training of Deep LSTM Networks

Momentum-based wavelet and double wavelet neural networks for power system applications

Text-independent speaker recognition using LSTM-RNN and speech enhancement

Comments

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Conference

Acceptance Rates

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Recommendations

Fast Training of Deep LSTM Networks

Momentum-based wavelet and double wavelet neural networks for power system applications

Text-independent speaker recognition using LSTM-RNN and speech enhancement

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations