research-article

Efficient digital neurons for large scale cortical architectures

Author:

James E. SmithAuthors Info & Claims

ISCA '14: Proceeding of the 41st annual international symposium on Computer architecuture

Pages 229 - 240

Published: 14 June 2014 Publication History

Abstract

Digital neurons are implemented with the goal of sup-porting research and development of architectures which implement the computational paradigm of the neocortex.

Four spiking digital neurons are implemented at the register transfer level in a manner that permits side-by-side comparisons. Two of the neurons contain two stages of ex-ponential decay, one for synapse conductances and one for membrane potential. The other two neurons contain only one stage of exponential decay for membrane potential.

The two stage neurons respond to an input spike with a change in membrane potential that has a non-infinite lead-ing edge slope; the one stage neurons exhibit a change in membrane potential with an abrupt, infinite leading edge slope. This leads to a behavioral difference when a number of input spikes occur in very close time proximity. However, the one stage neurons are as much as a factor of ten more energy efficient than the two stage neurons, as measured by the number of dynamic add-equivalent operations

A new two stage neuron is proposed. This neuron reduc-es the number of decay components and implements decays in both stages via piece-wise linear approximation. Togeth-er, these simplifications yield two stage neuron behavior with energy efficiency that is only about a factor of two worse than the simplest one stage neuron.

References

[1]

Ananthanarayanan, R., S. K. Esser, H. D. Simon, and D. S. Modha, "The Cat is Out of the Bag: Cortical Simulations with 109 Neurons, 1013 Synapses", IEEE Conference on High Performance Computing Networking, Storage and Analysis, pp. 1--12, 2009.

Digital Library

[2]

Arthur, J. V., P. A. Merolla, F. Akopyan, R. Alvarez, A. Cassidy, S. Chandra, S. Esser, N. Imamy, W. Risk, D. Rubin, R. Manohary, and D. Modha, "Building Block of a Programmable Neuromorphic Sub-strate: A Digital Neurosynaptic Core", 2012 International Joint Conference on Neural Networks, pp. 1--8, 2012.

[3]

Barbour, B., N. Brunel, V. Hakim, and J.-P. Nadal, "What Can We Learn from Synaptic Weight Distributions", TRENDS in Neurosciences, 30, no. 12, pp. 622--629, 2007.

[4]

Bohte, S. M., "The Evidence for Neural Information Processing with Precise Spike-times: A Survey", Natural Computing, 3, no. 2, pp. 195--206, 2004.

Digital Library

[5]

Brette, R., and W. Gerstner, "Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity," Journal of Neurophysiology, 94.5, pp. 3637--3642, 2005.

[6]

Brunel, N., "Persistent Activity and the Single-Cell Frequency-Current Curve in a Cortical Network Model", Network: Computation in Neural Systems, 11, no. 4 pp. 261--280, 2000.

[7]

Cassidy, A. S., P. Merolla, J. V. Arthur, S. K. Esser, B. Jackson, R. Alvarez-Icaza, P. Datta, J. Sawaday, T. M. Wong, V. Feldman, A. Amir, D. B.-D. Rubinx, F. Akopyan, E. McQuinn, W. P. Risk, and D. S. Modha, "Cognitive Computing Building Block: A Versatile and Efficient Digital Neuron Model for Neurosynaptic Cores", International Joint Conference on Neural Networks, 2013.

[8]

Deiss, S. R., R. J. Douglas, and A. M. Whatley, "A Pulse-Coded Communications Infrastructure for Neuromorphic Systems", Pulsed Neural Networks, pp. 157--178, 1999.

Digital Library

[9]

Emery, R., A. Yakovlev, and G. Chester, "Connection-Centric Network for Spiking Neural Networks", 3rd ACM/IEEE International Symposium on Networks-on-Chip, pp. 144--152, 2009.

Digital Library

[10]

Fidjeland, A. K., E. B. Roesch, M. P. Shanahan, W. Luk, "Nemo: A Platform For Neural Modelling of Spiking Neurons Using GPUs," Application-specific Systems, Architectures and Processors, 2009.

Digital Library

[11]

George, D., and J. Hawkins, "Towards A Mathematical Theory Of Cortical Micro-Circuits", PLoS Computational Biology 5.10, 2009.

[12]

Gerstner, W., H. Sprekeler, and G. Deco, "Theory and Simulation in Neuroscience", Science 338.6103, pp. 60--65, 2012.

[13]

Gerstner, W., and W. M. Kistler, Spiking Neuron Models: Single Neurons, Populations, Plasticity, Cambridge University Press, 2002.

Digital Library

[14]

Giacomo I., B. Linares-Barranco, T. J. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Häfliger, Sylvie Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. Saighi, T. Serrano-Gotarredona, J. Wijekoon, Y.Wang and K. Boahen, "Neuromorphic Silicon Neuron Circuits", Frontiers in Neuroscience 5, 2011.

[15]

Hawkins, J., S. Ahmad, and D. Dubinsky, "Hierarchical Temporal Memory Including HTM Cortical Learning Algorithms", Technical Report, Numenta, Inc, Palto Alto, 2010.

[16]

Hellmich, H. H., M. Geike, P. Griep, P. Mahr, M. Rafanelli, and H. Klar, "Emulation Engine for Spiking Neurons and Adaptive Synaptic Weights", IEEE International Joint Conference on Neural Networks, pp. 3261--3266, 2005.

[17]

Hodgkin, A., and A. Huxley, "A Quantitative Description of Membrane Current and Its Application to Conduction and Excitation in Nerve", The Journal of Physiology, 117.4, pp. 500--544, 1952.

[18]

Izhikevich, E. M., and G. M. Edelman, "Large-Scale Model Of Mammalian Thalamocortical Systems", Proceedings of The National Academy Of Science, 105.9, pp. 3593--3598, 2008.

[19]

Izhikevich, E. M, "Simple Model of Spiking Neurons", IEEE Transactions on Neural Networks, 14.6, pp. 1569--1572, 2003.

Digital Library

[20]

Izhikevich, E. M., J. Gally, and G. Edelman, "Spike-Timing Dynamics of Neuronal Groups", Cerebral Cortex, 14.8, pp. 933--944, 2004.

[21]

Izhikevich, E. M., "Polychronization: Computation with Spikes", Neural Computation, 18.2, pp. 245--282, 2006.

Digital Library

[22]

Jolivet, R., R. Kobayashi, A. Rauch, R. Naud, S. Shinomoto, W. Gerstner, "A Benchmark Test for a Quantitative Assessment of Simple Neuron Models", Journal of Neuroscience Methods, 169, pp. 417--424, 2008.

[23]

Joubert, A., B. Belhadj, O. Temam, and R. Héliot, "Hardware Spiking Neurons Design: Analog or Digital?", International Joint Conference on Neural Networks, pp. 1--5, 2012.

[24]

Kim, K.-H., S. Gaba, D. Wheeler, J. M. Cruz-Albrecht, T. Hussain, N. Srinivasa, and W. Lu, "A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic Applications", NanoLletters 12.1, pp. 389--395, 2011.

[25]

Kistler, W. M., W. Gerstner, and J. L. van Hemmen, "Reduction Of The Hodgkin-Huxley Equations to a Single-Variable Threshold Model", Neural Computation 9.5, pp. 1015--1045, 1997.

Digital Library

[26]

Lippmann, R., "An Introduction to Computing with Neural Nets", ASSP Magazine, IEEE 4.2, pp. 4--22, 1987.

[27]

Maass, W., "Networks of Spiking Neurons: The Third Generation of Neural Network Models," Neural Networks 10.9, pp.1659--1671, 1997.

Digital Library

[28]

Maass, W. "Computing with Spikes." Special Issue on Foundations of Information Processing of TELEMATIK 8.1 pp. 32--36, 2002.

[29]

Mainen, Z. F., and T. J. Sejnowski, "Reliability of Spike Timing in Neocortical Neurons", Science 268, pp. 1503--1506, 1995.

[30]

Markram, H., "The Blue Brain Project", Nature Reviews Neuroscience, 7, no. 2 pp. 153--160, 2006.

[31]

Nageswaran, J., N. Dutt, J. L. Krichmar, A. Nicolau, A. Veidenbaum, "Efficient Simulation of Large-Scale Spiking Neural Networks Using CUDA Graphics Processors", International Joint Conference on Neural Networks, 2009.

Digital Library

[32]

National Academy of Engineering, "Reverse-Engineer the Brain", http://www.engineeringchallenges.org/cms/8996/9109.aspx, 2012.

[33]

Nere, A., A. Hashmi, M. H. Lipasti, and G. Tononi, "Bridging the Semantic Gap: Emulating Biological Neuronal Behaviors with Simple Digital Neurons", HPCA, pp. 472--483. 2013.

Digital Library

[34]

Paugam-Moisy, H. and S. M. Bohte, "Computing with Spiking Neuron Networks," Handbook of Natural Computing, Springer, 2009.

[35]

Pospischil, M., Z. Piwkowska, T. Bal, A. Destexhe, "Comparison of Different Neuron Models to Conductance-Based Post-Stimulus Time Histograms Obtained in Cortical Pyramidal Cells Using Dynamic-Clamp in Vitro", Biological cybernetics 105.2, pp.167--180, 2011.

Digital Library

[36]

Rast, A. D., M. Khan, X. Jin, L. A. Plana, and S. B. Furber, "A Universal Abstract-Time Platform for Real-Time Neural Networks", The Relevance of the Time Domain to Neural Network Models. Springer US, 12, pp.135--157, 2012.

[37]

Rauch, A., G. La Camera, H. R. Luscher, W. Senn, and S. Fusi, "Neocortical Pyramidal Cells Respond as Integrate-And-Fire Neurons to in Vivo-Like Input Currents," Journal of neurophysiology 90, no. 3 pp. 1598--1612, 2003.

[38]

Schemmel, Johannes, D. Bruderle, A. Grubl, M. Hock, K. Meier, and S. Millner, "A Wafer-Scale Neuromorphic Hardware System for Large-Scale Neural Modeling", International Symposium on Circuits and Systems, pp. 1947--1950, 2010.

[39]

Seo, J.-S., B. Brezzo, Y. Liu, B. D. Parker, S. K. Esser, R. K. Montoye, B. Rajendran. J. A. Tierno, L. Chang, D. S. Modha, and D. J. Friedman, "A 45nm CMOS Neuromorphic Chip with a Scalable Architecture for Learning in Networks of Spiking Neurons", Custom Integrated Circuits Conference, pp. 1--4, 2011.

[40]

Song, S., P. J. Sjöström, M. Reigl, S. Nelson, and D. B. Chklovskii, "Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits", PLoS Biology, 3(3), 2005.

[41]

Stein, R. B. "A Theoretical Analysis Of Neuronal Variability", Biophysical Journal, 5.2, pp. 173--194, 1965.

[42]

Thorpe, Simon J., and M. Imbert. "Biological Constraints on Connectionist Modelling", Connectionism in perspective pp. 63--92, 1989.

[43]

Upegui, A., C. A. Peña-Reyes, and E. Sanchez, "An FPGA Platform For On-Line Topology Exploration of Spiking Neural Networks", Microprocessors and Microsystems, 29.5, pp. 211--223, 2005.

[44]

Vogels, T. P., H. Sprekeler, F. Zenke, C. Clopath, and W.Gerstner, "Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks", Science 334.6062, pp.1569--1573, 2011

[45]

Vogels, T. P., and L. F. Abbott, "Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons," The Journal of Neuroscience 25.46 pp. 10786--10795, 2005.

Cited By

Satyanarayanan MGao WLucia BWolman AZhong L(2019)The Computing Landscape of the 21st CenturyProceedings of the 20th International Workshop on Mobile Computing Systems and Applications10.1145/3301293.3302357(45-50)Online publication date: 22-Feb-2019
https://dl.acm.org/doi/10.1145/3301293.3302357
Lee DLee GKwon DLee SKim YKim J(2018)FlexonProceedings of the 45th Annual International Symposium on Computer Architecture10.1109/ISCA.2018.00032(275-288)Online publication date: 2-Jun-2018
https://dl.acm.org/doi/10.1109/ISCA.2018.00032
Bhattacharjee AHunter HMoreno JEmer JSanchez D(2017)Using branch predictors to predict brain activity in brain-machine implantsProceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3123939.3123943(409-422)Online publication date: 14-Oct-2017
https://dl.acm.org/doi/10.1145/3123939.3123943
Show More Cited By

Recommendations

Efficient digital neurons for large scale cortical architectures
ISCA '14

Digital neurons are implemented with the goal of sup-porting research and development of architectures which implement the computational paradigm of the neocortex.

Four spiking digital neurons are implemented at the register transfer level in a manner ...
Inhibitory conductance dynamics in cortical neurons during activated states

During activated states in vivo, neocortical neurons are subject to intense synaptic activity and high-amplitude membrane potential (V"m) fluctuations. These ''high-conductance'' states may strongly affect the integrative properties of cortical neurons. ...
Sensory experience modifies spontaneous state dynamics in a large-scale barrel cortical model

Experimental evidence suggests that spontaneous neuronal activity may shape and be shaped by sensory experience. However, we lack information on how sensory experience modulates the underlying synaptic dynamics and how such modulation influences the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISCA '14: Proceeding of the 41st annual international symposium on Computer architecuture

June 2014

566 pages

ISBN:9781479943944

General Chairs:
Pen-Chung Yew
University of Minnesota
,
Antonia Zhai
University of Minnesota
,
Program Chair:
Steve Keckler
NVIDIA/University of Texas at Austin

ACM SIGARCH Computer Architecture News Volume 42, Issue 3
ISCA '14
June 2014
552 pages
ISSN:0163-5964
DOI:10.1145/2678373
Editor:
Doug DeGroot
acm dot org
Issue’s Table of Contents

Sponsors

IEEE TCCA: IEEE Computer Society Technical Committee on Computer Architecture
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

IEEE Press

Publication History

Published: 14 June 2014

Check for updates

Qualifiers

Research-article

Conference

ISCA'14

Sponsor:

IEEE TCCA
SIGARCH

ISCA'14: The 41st Annual International Symposium on Computer Architecture

June 14 - 18, 2014

Minnesota, Minneapolis, USA

Acceptance Rates

Overall Acceptance Rate 543 of 3,203 submissions, 17%

Upcoming Conference

ISCA '25

Sponsor:
sigarch

The 52nd Annual International Symposium on Computer Architecture

June 21 - 25, 2025

Tokyo , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
715
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Satyanarayanan MGao WLucia BWolman AZhong L(2019)The Computing Landscape of the 21st CenturyProceedings of the 20th International Workshop on Mobile Computing Systems and Applications10.1145/3301293.3302357(45-50)Online publication date: 22-Feb-2019
https://dl.acm.org/doi/10.1145/3301293.3302357
Lee DLee GKwon DLee SKim YKim J(2018)FlexonProceedings of the 45th Annual International Symposium on Computer Architecture10.1109/ISCA.2018.00032(275-288)Online publication date: 2-Jun-2018
https://dl.acm.org/doi/10.1109/ISCA.2018.00032
Bhattacharjee AHunter HMoreno JEmer JSanchez D(2017)Using branch predictors to predict brain activity in brain-machine implantsProceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3123939.3123943(409-422)Online publication date: 14-Oct-2017
https://dl.acm.org/doi/10.1145/3123939.3123943
Du ZBen-Dayan Rubin DChen YHe LChen TZhang LWu CTemam OPrvulovic M(2015)Neuromorphic acceleratorsProceedings of the 48th International Symposium on Microarchitecture10.1145/2830772.2830789(494-507)Online publication date: 5-Dec-2015
https://dl.acm.org/doi/10.1145/2830772.2830789
Liu XPu ZQu PZheng WZhang Y(2024)ActiveN: A Scalable and Flexibly-Programmable Event-Driven Neuromorphic Processor2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00085(1122-1137)Online publication date: 2-Nov-2024
https://doi.org/10.1109/MICRO61859.2024.00085
Yao XLi FMo ZCheng JKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)GLIFProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602600(32160-32171)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602600
Lee HKim CChung YKim JSherwood TBerger EKozyrakis C(2021)NeuroEngine: a hardware-based event-driven simulation system for advanced brain-inspired computingProceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3445814.3446738(975-989)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3445814.3446738

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten