research-article

Improved integrate-and-fire neuron models for inference acceleration of spiking neural networks

Authors:

Yongcheng Zhou,

Anguo ZhangAuthors Info & Claims

Applied Intelligence, Volume 51, Issue 4

Pages 2393 - 2405

https://doi.org/10.1007/s10489-020-02017-3

Published: 01 April 2021 Publication History

Abstract

We study the effects of different bio-synaptic membrane potential mechanisms on the inference speed of both spiking feed-forward neural networks and spiking convolutional neural networks. These mechanisms are inspired by biological neuron phenomena include electronic conduction in neurons and chemical neurotransmitter attenuation between presynaptic and postsynaptic neurons. In the area of spiking neural networks, we model some biological neural membrane potential updating strategies based on integrate-and-fire (I&F) spiking neurons. These include the spiking neuron model with membrane potential decay (MemDec), the spiking neuron model with synaptic input current superposition at spiking time (SynSup), and the spiking neuron model with synaptic input current accumulation (SynAcc). Experiment results show that compared with the general I&F model (one of the most commonly used spiking neuron models), SynSup and SynAcc can effectively improve the spiking inference speed of spiking feed-forward neural networks and spiking convolutional neural networks.

References

[1]

Bodyanskiy Y, Dolotov A, Pliss I, Malyar M (2016) A fast learning algorithm of self-learning spiking neural network. In: IEEE international conference on data stream mining & processing, pp 104–107.

[2]

Bohte SM, Kok JN, and Poutre HL Error-backpropagation in temporally encoded networks of spiking neurons Neurocomputing 2002 48 1 17-37

[3]

Camunas-Mesa L, Zamarreno-Ramos C, Linares-barranco A, et al. An event-driven multi-kernel convolution processor module for event-driven vision sensors IEEE Journal of Solid-State Circuits 2012 47 2 504-517

[4]

Cao Y and Chen Y Spiking deep convolutional neural networks for energy-efficient object recognition Int J Comput Vis 2014 113 1 54-66

[5]

Carlson KD, Nageswaran JM, Dutt N, Krichmar JL (2014) An efficient automated parameter tuning framework for spiking neural networks. Front Neurosci 8.

[6]

Diehl PU, Neil D, Binas J, Cook M, Liu SC, Pfeiffer M (2015) Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International joint conference on neural networks.

[7]

Fang H, Wang Y, and He J Spiking neural network for cortical neuronal spike train decoding Neural Comput 2010 22 4 1060-1085

[8]

Gutig R and Sompolinsky H The tempotron: a neuron that learns spike timing-based decisions Nat Neurosci 2006 9 3 420-428

[9]

Huang S, Rozas C, et al. Associate hebbian synaptic plasticity in primate visual cortex J Neurosci 2014 34 22 7575-7579

[10]

Humphries MD and Gurney K Solution methods for a new class of simple model neurons Neural Comput 2007 19 12 3216-3225

[11]

Johnson AP, Liu J, Millard A, et al. (2017) Homeostatic fault tolerance in spiking neural networks: A dynamic hardware perspective. IEEE Transactions on Circuits and Systems-I: Regular Papers, pp 1–13.

[12]

Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE International Conference on Neural Networks, pp 4

[13]

Lehky SR Decoding poisson spike trains by gaussian filtering Neural Comput 2010 22 5 1245-1271

[14]

Lin Z, Shen J, Ma D, and Meng J Quantisation and pooling method for low-inference-latency spiking neural networks Electron Lett 2017 53 20 1347-1348

[15]

Maass W Networks of spiking neurons: The third generation of neural network models Neural Netw 1997 9 10 1659-1672

[16]

Masquelier T and Thorpe SJ Unsupervised learning of visual features through spike timing dependent plasticity PLoS Comput Biol 2007 3 2 e31

[17]

McMahon DB and Leoplod DA Stimuls timing-dependent plasticity in high-level vision Curr Biol 2012 22 4 332-337

[18]

Meliza CD and Dan Y Receptive-field modification in rat visual cortex induced by paired visual stimulation and single-cell spiking Neuron 2006 49 2 183-189

[19]

Merolla P, Arthur J, Akopyan F, et al. (2011) A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm. IEEE Custom Integrated Circuits Conference.

[20]

O’Connor P, Neil D, Liu SC, Delbruck T, Pfeiffer M (2013) Real-time classification and sensor fusion with a spiking deep belief network. Front Neurosci 7.

[21]

Rossum V, W MC, Turrigiano GG, and Nelson SB Fast propagation of firing rates through layered networks of noisy neurons J Neurosci 2002 22 5 1956-1966

[22]

Rueckauer B, Lungu IA, Hu Y, Pfeiffer M (2016) Theory and tools for the conversion of analog to spiking convolutional neural networks. arXiv:1612.04052 [cs, stat]

[23]

Schrauwen B and Campenhout JV Improving spike-prop: Enhancements to an error-backpropagation rule for spiking neural networks Proceedings of ProRISC Workshop 2004 11 301-305

[24]

Skocik MJ and Long LN On the capabilities and computational costs of neuron models IEEE Trans Neural Netw Learn Syst 2014 25 8 1474-1483

[25]

Tino P and Mills AJS Learning beyond finite memory in recurrent networks of spiking neurons Neural Comput 2006 18 3 591-613

[26]

Ventura V and Todorova S A computationally efficient method for incorporating spike waveform information into decoding algorithms Neural Comput 2015 25 5 1033-1050

[27]

Zhang A, Zhou H, Li X, and Zhu W Fast and robust learning in Spiking Feed-forward Neural Networks based on Intrinsic Plasticity mechanism Neurocomputing 2019 365 102-112

[28]

Zhang S, Zhang A, Ma Y, and Zhu W Intrinsic plasticity based inference acceleration for spiking Multi-Layer perceptron IEEE Access 2019 7 73685-73693

Cited By

Liang JWang CMa DLi RYue KLi W(2024)Learning improvement of spiking neural networks with dynamic adaptive hyperparameter neuronsApplied Intelligence10.1007/s10489-024-05629-154:19(9158-9176)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10489-024-05629-1
Guo LZhao QWu YXu G(2022)Small-world spiking neural network with anti-interference ability based on speech recognition under interferenceApplied Soft Computing10.1016/j.asoc.2022.109645130:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.asoc.2022.109645

Recommendations

Improved Izhikevich neurons for spiking neural networks

Spiking neural networks constitute a modern neural network paradigm that overlaps machine learning and computational neurosciences. Spiking neural networks use neuron models that possess a great degree of biological realism. The most realistic model of ...
Effects of Noise on Leaky Integrate-and-Fire Neuron Models for Neuromorphic Computing Applications
Computational Science and Its Applications – ICCSA 2022
Abstract
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), ...
A spiking neuron model: applications and learning

This paper presents a biologically inspired, hardware-realisable spiking neuron model, which we call the Temporal Noisy-Leaky Integrator (TNLI). The dynamic applications of the model as well as its applications in Computational Neuroscience are ...

Comments

Information & Contributors

Information

Published In

cover image Applied Intelligence

Applied Intelligence Volume 51, Issue 4

Apr 2021

874 pages

ISSN:0924-669X

Issue’s Table of Contents

© Springer Science+Business Media, LLC, part of Springer Nature 2020.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 April 2021

Accepted: 09 October 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liang JWang CMa DLi RYue KLi W(2024)Learning improvement of spiking neural networks with dynamic adaptive hyperparameter neuronsApplied Intelligence10.1007/s10489-024-05629-154:19(9158-9176)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10489-024-05629-1
Guo LZhao QWu YXu G(2022)Small-world spiking neural network with anti-interference ability based on speech recognition under interferenceApplied Soft Computing10.1016/j.asoc.2022.109645130:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.asoc.2022.109645

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents