High-Accuracy Spiking Neural Network for Objective Recognition Based on Proportional Attenuating Neuron
Authors: 
Li-Ye Niu, Ying Wei, Jun-Yu Long, Wen-Bo LiuAuthors Info & Claims
Li-Ye Niu, Ying Wei, Jun-Yu Long, Wen-Bo LiuAuthors Info & ClaimsPages 1055 - 1073
Abstract
Spiking neural network (SNN) is one of the most successful methods to imitate biological brain behavior and learning potential. The information processing mechanism of SNN combines the concepts of time and space. To address the performance dropping problem during the conversion process from the artificial neural network (ANN) to SNN, this paper proposes a proportional attenuation leaky integrate-and-fire (RA-LIF) neuron model to solve the problem of membrane potential loss caused by leaky integrate-and-fire (LIF) neurons. The accuracy of the SNN network based on RA-LIF neurons on MNIST is 98.76%. This paper also proposes a weight normalization method to help adjust the network spiking rate to reduce the loss. We evaluate and analyze the performance of SNN networks based on LIF, RA-LIF, and AD-LIF. By analyzing the spike firing rate and convergence rate, the effects of spike frequency and neuron threshold on the performance of the network are discussed. So far, the SNN transformed by ANN can only achieve worse or similar performance compared with the original network. For the first time, the performance of the SNN transformed by the proposed RA-LIF neuron model and weight normalization method is better than the original network, which shows the test accuracy of 98.88% on MNIST. The SNN obtained by converting the LeNet network using the above method achieves a test accuracy of 98.91% on MNIST. In conclusion, the RA-LIF neuron model and normalization method proposed in this paper have promising applicability. Moreover, this method has the characteristics of high precision and fast calculation speed, which can provide a reference for the research of the SNN framework.
References
[1]
Tavanaei A, Ghodrati M, Kheradpisheh SR, et al. Deep learning in spiking neural networks Neural Netw 2019 111 47-63
[2]
Butts DA, Weng C, Jin J, et al. Temporal precision in the neural code and the timescales of natural vision Nature 2007 449 7158 92-95
[3]
Lin XH, Wang XW, and Hao ZJ Supervised learning in multilayer spiking neural networks with inner products of spike trains Neurocomputing 2017 237 59-70
[4]
Zhao B, Ding R, Chen S, et al. Feedforward categorization on AER motion events using cortex-like features in a spiking neural network IEEE Trans Neural Netw learn Syst 2014 26 9 1963-1978
[5]
Mozafari M, Ganjtabesh M, Nowzari-Dalini A, et al. Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks Pattern Recogn 2019 94 87-95
[6]
Lee C, Panda P, Srinivasan G, et al. Training deep spiking convolutional neural networks with stdp-based unsupervised pre-training followed by supervised fine-tuning Front Neurosci 2018 12 435
[7]
Li H, Li G, Ji X, et al. Deep representation via convolutional neural network for classification of spatiotemporal event streams Neurocomputing 2018 299 1-9
[8]
Lee JH, Delbruck T, and Pfeiffer M Training deep spiking neural networks using backpropagation Front Neurosci 2016 10 508
[9]
Querlioz D, Bichler O, Dollfus P, et al. Immunity to device variations in a spiking neural network with memristive nanodevices IEEE Trans Nanotechnol 2013 12 3 288-295
[10]
Neil D, Pfeiffer M, Liu S (2016) Learning to be efficient: algorithms for training low-latency, low-compute deep spiking neural networks. In: ACM Symposium on Applied Computing (SAC). Pisa: ACM, 293~298
[11]
Xu Q, Qi Y, Yu H, et al (2018) CSNN: An Augmented Spiking based Framework with Perceptron-Inception. In: International Joint Conference on Artificial Intelligence. Stockholm: Morgan Kaufmann, 1646~1652
[12]
Cao Y, Chen Y, and Khosla D Spiking deep convolutional neural networks for energy-efficient object recognition Int J Comput Vision 2015 113 1 54-66
[13]
Diehl PU, Neil D, Binas J et al (2015) Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International Joint Conference on Neural Networks. Killarney: IEEE, 1~8
[14]
Danelljan M, Shahbaz Khan F, Felsberg M, et al (2014) Adaptive color attributes for real-time visual tracking. In: Computer Vision and Pattern Recognition. Columbus: IEEE, 1090~1097.
[15]
Ma J, Shao W, Ye H, et al. Arbitrary-oriented scene text detection via rotation proposals IEEE Trans Multimedia 2018 20 11 3111-3122
[16]
Qiang FA and Hd A An ensemble unsupervised spiking neural network for objective recognition- ScienceDirect Neurocomputing 2021 419 47-58
[17]
Song Z, Wu X, Yuan M, et al. An unsupervised spiking deep neural network for object recognition 2019 Cham Springer
[18]
Hamidinekoo A, Denton E, Rampun A, et al (2018) Deep learning in mammography and breast histology, an overview and future trends. Medical Image Analysis, 45–67
[19]
Urbanczik R and Senn W Learning by the dendritic prediction of somatic spiking Neuron 2014 81 521-528
[20]
Burkitt AN A Review of the integrate-and-fire neuron model: I Homogeneous synaptic input Biol Cybern 2006 95 1 1-19
[21]
Markram H and Gerstner W Spike-timing-dependent plasticity: a comprehensive overview Front Synaptic Neurosci 2012 4 2 2
[22]
Bienenstock EL, Cooper LN, and Munro PW Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex[J] J Neurosci 1982 2 1 32-48
[23]
Mishra D, Yadav A, and Kalra PK A learning algorithm for a novel neural network architecture motivated by Integrate-and-Fire Neuron Model Neural Network World 2006 16 6 513-532
[24]
Aniello B, Luigia C, Enrica P, and Maria FC A leaky integrate-and-fire model with adaptation for the generation of a spike train Math Biosci Eng 2017 13 3 483-493
[25]
Brette R Adaptive exponential integrate-and-fire model as an effective description of neuronal activity J Neurophysiol 2005 94 5 3637
[26]
Jang V Inversion of Fredholm integral equations of the first kind with fully connected neural networks J Franklin Inst 1992 329 2 241-257
[27]
Yang X, Zhang Z, Zhu W, et al. Deterministic conversion rule for CNNs to efficient spiking convolutional neural networks Science China Inf Sci 2020 63 2 1-19
[28]
Palm R B (2012) Prediction as a Candidate for Learning Deep Hierarchical Models of Data. Technical University of Denmark
[29]
Lee JH, Delbruck T, Pfeiffer M (2016) Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10
[30]
Peter, U, Diehl, et al (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci 9:99
[31]
Cohen G K, Garrick O, Leng SH, et al (2016) Skimming Digits: Neuromorphic Classification of Spike-Encoded Images. Frontiers in Neuroscience, 10(184).
[32]
Neftci E, Das S, Pedroni B, et al. Event-driven contrastive divergence for spiking neuromorphic systems Front Neurosci 2013 7 8 272
Index Terms
- High-Accuracy Spiking Neural Network for Objective Recognition Based on Proportional Attenuating Neuron
Index terms have been assigned to the content through auto-classification.
Recommendations
Improved integrate-and-fire neuron models for inference acceleration of spiking neural networks
AbstractWe study the effects of different bio-synaptic membrane potential mechanisms on the inference speed of both spiking feed-forward neural networks and spiking convolutional neural networks. These mechanisms are inspired by biological neuron ...
Supervised associative learning in spiking neural network
ICANN'10: Proceedings of the 20th international conference on Artificial neural networks: Part IIn this paper, we propose a simple supervised associative learning approach for spiking neural networks. In an excitatory-inhibitory network paradigm with Izhikevich spiking neurons, synaptic plasticity is implemented on excitatory to excitatory ...
Improved Izhikevich neurons for spiking neural networks
Spiking neural networks constitute a modern neural network paradigm that overlaps machine learning and computational neurosciences. Spiking neural networks use neuron models that possess a great degree of biological realism. The most realistic model of ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 April 2022
Accepted: 13 October 2021
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Nature Science Foundation of China
- Key R & D projects of Liaoning Province, 460 China
- the Open Project Program Foundation of the Key Laboratory of Opto-Electronics Information Processing, Chinese Academy of Sciences
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Dec 2024