research-article

Spike-Time-Dependent Encoding for Neuromorphic Processors

Authors:

Bryant T. Wysocki,

Clare D. Thiem,

Nathan R. McDonald,

Yang YiAuthors Info & Claims

ACM Journal on Emerging Technologies in Computing Systems (JETC), Volume 12, Issue 3

Article No.: 23, Pages 1 - 21

https://doi.org/10.1145/2738040

Published: 21 September 2015 Publication History

Abstract

This article presents our research towards developing novel and fundamental methodologies for data representation using spike-timing-dependent encoding. Time encoding efficiently maps a signal's amplitude information into a spike time sequence that represents the input data and offers perfect recovery for band-limited stimuli. In this article, we pattern the neural activities across multiple timescales and encode the sensory information using time-dependent temporal scales. The spike encoding methodologies for autonomous classification of time-series signatures are explored using near-chaotic reservoir computing. The proposed spiking neuron is compact, low power, and robust. A hardware implementation of these results is expected to produce an agile hardware implementation of time encoding as a signal conditioner for dynamical neural processor designs.

References

[1]

L. F. Abbott. 1999. Lapicque's introduction of the integrate-and-fire model neuron (1907). Brain Res. Bull. 50, 303--304.

[2]

P. E. Allen and D. R. Holberg. 2002. CMOS Analog Circuit Design. Oxford University Press.

[3]

J. V. Arthur and K. Boahen. 2011. Silicon-neuron design: A dynamical systems approach. IEEE Trans. Circ. Syst. 58, 1034--1043.

[4]

T. Asai, Y. Kanazawa, and Y. Amemiya. 2003. A subthreshold MOS neuron circuit based on the Volterra system. IEEE Trans. Neural Netw. 14, 1308--1312.

Digital Library

[5]

R. J. Baker. 2011. CMOS: Circuit Design, Layout, and Simulation. John Wiley and Sons.

Digital Library

[6]

A. Bindal and S. Hamedi-Hagh. 2007. The design of a new spiking neuron using dual work function silicon nanowire transistors. Nanotechnol. 18, 9.

[7]

L. Binstock and H. Lecar. 1969. Ammonium ion currents in the squid giant axon. J. General Physiol. 53, 342--361.

[8]

R. Brasselet, S. Panzeri, N. K. Logothetis, and C. Kayser. 2012. Neurons with stereotyped and rapid responses provide a reference frame for relative temporal coding in primate auditory cortex. J. Neurosci. 32, 2998--3008.

[9]

M. A. B. Brazier. 1951. The Electrical Activity of the Nervous System: A Textbook for Students. Pitman.

[10]

R. Brette and W. Gerstner. 2005. Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J. Neurophysiol. 94, 3637--3642.

[11]

A. N. Burkitt. 2006. A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input. Biologic. Cybernet. 95, 1--19.

Digital Library

[12]

A. S. Cassidy, J. Georgiou, and A. G. Andreou. 2013. Design of silicon brains in the nano-CMOS era: Spiking neurons, learning synapses and neural architecture optimization. Neural Netw. 45, 4--26.

Digital Library

[13]

Y.-K. Chang, C.-S. Wang, and C.-K. Wang. 2007. A 8-bit 500-KS/s low power SAR ADC for bio-medical applications. In Proceedings of the IEEE Asian Solid-State Circuits Conference (ASSCC'07). 228--231.

[14]

S. M. Chase and E. D. Young. 2007. First-spike latency information in single neurons increases when referenced to population onset. Proc. Nat. Acad. Sci. 104, 5175--5180.

[15]

K. S. Cole and A. L. Hodgkin. 1939. Membrane and protoplasm resistance in the squid giant axon. J. General Physiol. 22, 671--687.

[16]

E. Culurciello and A. Andreou. 2003. An 8-bit, 1mW successive approximation ADC in SOI CMOS. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS'03). 301--304.

[17]

P. Dautriche. 2011. Analog design trends and challenges in 28 and 20nm CMOS technology. In Proceedings of the European Solid-State Device Research Conference (ESSCIRC'11). 1--4.

[18]

N. Ding and J. Z. Simon. 2012. Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. J. Neurophysiol. 107, 78--89.

[19]

I. E. Ebong and P. Mazumder. 2012. CMOS and memristor-based neural network design for position detection. Proc. IEEE 100, 2050--2060.

[20]

R. Fitzhugh. 1961. Impulses and physiological states in theoretical models of nerve membrane. Biophys. J. 1, 445--466.

[21]

A. K. Friesz, A. C. Parker, C. Zhou, K. Ryu, J. M. Sanders, H.-S. P. Wong, and J. Deng. 2007. A biomimetic carbon nanotube synapse circuit. In Proceedings of the Biomedical Engineering Society Annual Fall Meeting (BMES'07).

[22]

W. Gerstner and W. M. Kistler. 2002. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press.

Digital Library

[23]

D. J. Griffiths and R. College. 1999. Introduction to Electrodynamics. Prentice Hall, Upper Saddle River, NJ.

[24]

X. Guo, X. Qi, and J. G. Harris. 2007. A time-to-first-spike CMOS image sensor. IEEE Sens. J. 7, 1165.

[25]

I. S. Han. 2006. Mixed-signal neuron-synapse implementation for large-scale neural network. Neurocomput. 69, 1860--1867.

[26]

P. J. Harpe, C. Zhou, Y. Bi, N. P. Van Der Meijs, X. Wang, K. Philips, G. Dolmans, and H. De Groot. 2011. A 26 W 8 bit 10 MS/s asynchronous SAR ADC for low energy radios. IEEE J. Solid-State Circ. 46, 1585--1595.

[27]

R. M. Harris-Warrick. 1992. Dynamic Biological Networks: The Stomatogastric Nervous System. MIT Press.

[28]

W. Himmelbauer and A. G. Andreou. 1997. Log-domain circuits in subthreshold MOS. In Proceedings of the 40^th IEEE Midwest Symposium on Circuits and Systems (MWSCAS'97). 26--30.

[29]

A. L. Hodgkin and A. F. Huxley. 1952. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117, 500.

[30]

M. Hu, H. Li, Q. Wu, and G. S. Rose. 2012. Hardware realization of BSB recall function using memristor crossbar arrays. In Proceedings of the 49^th Annual Design Automation Conference (DAC'12). 498--503.

Digital Library

[31]

G. Indiveri. 2003. A low-power adaptive integrate-and-fire neuron circuit. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'04). 820--823.

[32]

G. Indiveri, E. Chicca, and R. J. Douglas. 2009. Artificial cognitive systems: From VLSI networks of spiking neurons to neuromorphic cognition. Cogn. Comput. 1, 119--127.

[33]

G. Indiveri and T. K. Horiuchi. 2011. Frontiers in neuromorphic engineering. Frontiers Neurosci. 5, 118.

[34]

G. Indiveri, B. Linares-Barranco, T. J. Hamilton, A. Van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Häfliger, and S. Renaud. 2011. Neuromorphic silicon neuron circuits. Frontiers Neurosci. 5, 73.

[35]

H. Jaeger. 2001. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. German Nat. Res. Center Inf. Technol. 148, 34.

[36]

O. Jensen, B. Gips, T. O. Bergmann, and M. Bonnefond. 2014. Temporal coding organized by coupled alpha and gamma oscillations prioritize visual processing. Trends Neurosci. 37, 7, 357--369.

[37]

A. Joubert, B. Belhadj, and R. Héliot. 2011. A robust and compact 65 nm LIF analog neuron for computational purposes. In Proceedings of the 9^th IEEE International Conference on New Circuits and Systems (NEWCAS'11). 9--12.

[38]

A. Joubert, B. Belhadj, O. Temam, and R. Heliot. 2012. Hardware spiking neurons design: Analog or digital? In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN'12). 1--5.

[39]

B. Katz and B. Katz. 1966. Nerve, Muscle, and Synapse. McGraw-Hill, New York.

[40]

Y. Kim, Y. Zhang, and P. Li. 2012. A digital neuromorphic VLSI architecture with memristor crossbar synaptic array for machine learning. In Proceedings of the IEEE International SOC Conference (SOCC'12). 328--333.

[41]

T. Le-Ngoc and F. Trans. 2005. Channel equalization system and method. https://www.google.com/patents/US20030016770.

[42]

Y. Y. Lee, S. Harwell, S. Khurshid, and D. Marinov. 2013. Temporal code completion and navigation. In Proceedings of the International Conference on Software Engineering (ICSE'13). 1181--1184.

Digital Library

[43]

Y. Li and L. He. 2007. First-order continuous-time sigma-delta modulator. In Proceedings of the International Symposium on Quality Electronic Design (ISQED'07). 229--232.

Digital Library

[44]

B. Liu, Y. Chen, B. Wysocki, and T. Huang. 2012. The circuit realization of a neuromorphic computing system with memristor-based synapse design. In Neural Information Processing, Springer, 357--365.

Digital Library

[45]

Y.-H. Liu and X.-J. Wang. 2001. Spike-frequency adaptation of a generalized leaky integrate-and-fire model neuron. J. Comput. Neurosci. 10, 25--45.

[46]

W. Maass and H. Markram. 2004. On the computational power of circuits of spiking neurons. J. Comput. Syst. Sci. 69, 593--616.

Digital Library

[47]

C. Mead and M. Ismail. 1989. Analog VLSI Implementation of Neural Systems. Springer.

[48]

S. Millner, A. Grübl, K. Meier, J. Schemmel, and M.-O. Schwartz. 2010. A VLSI implementation of the adaptive exponential integrate-and-fire neuron model. http://papers.nips.cc/paper/3995-a-vlsi-implementation-of-the-adaptive-exponential-integrate-and-fire-neuron-model.pdf.

[49]

L. Paninski, J. W. Pillow, and E. P. Simoncelli. 2004. Maximum likelihood estimation of a stochastic integrate-and-fire neural encoding model. Neural Comput. 16, 2533--2561.

Digital Library

[50]

S. Panzeri, N. Brunel, N. K. Logothetis, and C. Kayser. 2010. Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 33, 111--120.

[51]

V. Patil and S. Shimpi. 2011. Handwritten English character recognition using neural network. Elixir Comput. Sci. Engin. 41, 5587--5591.

[52]

X. Qi, X. Guo, and J. G. Harris. 2004. A time-to-first spike CMOS imager. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'04). 824--827.

[53]

R. Q. Quiroga and S. Panzeri. 2013. Principles of Neural Coding. CRC Press.

[54]

B. Razavi. 2002. Design of Analog CMOS Integrated Circuits. Tata McGraw-Hill Education.

Digital Library

[55]

D. S. Reich, F. Mechler, K. P. Purpura, and J. D. Victor. 2000. Interspike intervals, receptive fields, and information encoding in primary visual cortex. The J. Neurosci. 20, 1964--1974.

[56]

R. Sarpeshkar. 1998. Analog versus digital: Extrapolating from electronics to neurobiology. Neural Comput. 10, 1601--1638.

Digital Library

[57]

D. J. Sauer. 1996. Wide frequency range CMOS relaxation oscillator with variable hysteresis. http://www.google.dj/patents/US5497127.

[58]

M. C. Schneider and C. Galup-Montoro. 2010. CMOS Analog Design Using All-Region MOSFET Modeling. Cambridge University Press.

Digital Library

[59]

B. Schrauwen, D. Verstraeten, and J. Van Campenhout. 2007. An overview of reservoir computing: Theory, applications and implementations. In Proceedings of the 15^th European Symposium on Artificial Neural Networks (ESANN'07). 471--482.

Digital Library

[60]

R. Z. Shi and T. K. Horiuchi. 2003. A summating, exponentially-decaying CMOS synapse for spiking neural systems. In Proceedings of the Conference on Neural Information Processing Systems (NIPS'03).

[61]

C. Shoushun and A. Bermak. 2005. A low power CMOS imager based on time-to-first-spike encoding and fair AER. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS'05). 5306--5309.

[62]

J. Sikarwar and R. Gurjar. 2012. A low-cost first-order sigma-delta converter design and analysis in 0.18 μ m technology. http://www.ijera.com/papers/Vol2_issue1/DD21668671.pdf.

[63]

K. Ueno. 2010. CMOS Voltage and Current Reference Circuits Consisting of Subthreshold MOSFETs. Solid State Circuits Technologies. Intech Press.

[64]

K. Ueno, T. Hirose, T. Asai, and Y. Amemiya. 2010. A 1-600-current reference circuit consisting of subthreshold CMOS circuits. IEEE Trans. Circ. Syst. 57, 681--685.

Digital Library

[65]

A. Van Schaik. 2001. Building blocks for electronic spiking neural networks. Neural Netw. 14, 617--628.

[66]

D. Wei and J. G. Harris. 2004. Signal reconstruction from spiking neuron models. In Proceedings of the International Symposium on Circuits and Systems (ISCAS'04). 353--356.

[67]

N. Weste and D. Harris. 2005. CMOS VLSI Design. A Circuits and Systems Perspective. Pearson.

Digital Library

[68]

J. H. Wijekoon and P. Dudek. 2008a. Compact silicon neuron circuit with spiking and bursting behaviour. Neural Netw. 21, 524--534.

Digital Library

[69]

J. H. Wijekoon and P. Dudek. 2008b. Integrated circuit implementation of a cortical neuron. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS'08). 1784--1787.

[70]

R. Wojtyna and T. Talaska. 2006. Transresistance CMOS neuron for adaptive neural networks implemented in hardware. Bull. Polish Acad. Sci. Tech. Sci. 54, 4.

[71]

H. Wu, B. Li, M. Zou, W. Huang, and Y. Wang. 2011. An 1.2 V 8-bit 1-MS/s single-input res-cap segment SAR ADC for temperature sensor in LTE. In Proceedings of the IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC'11). 1--2.

[72]

M. L. Ya, S. Khan, A. N. Nordin, A. Z. Alam, J. Omar, K. A. S. Al-Khateeb, M. R. Islam, and A. W. Naji. 2011. A low-cost first-order sigma-delta converter design and analysis. In Proceedings of the Instrumentation and Measurement Technology Conference (I2MTC'11). 1--5.

[73]

F. M. Yaul and A. P. Chandrakasan. 2014. 11.3 A 10b 0.6 nW SAR ADC with data-dependent energy savings using LSB-first successive approximation. In Proceedings of the IEEE International Solid-State Circuits Conference; Digest of Technical Papers (ISSCC'14). 198--199.

[74]

K. Yeh, C.-W. Chen, D. Lo, and K. F. Liu. 2012. Neural-network fuzzy control for chaotic tuned mass damper systems with time delays. J. Vibrat. Control 18, 785--795.

[75]

B. Yuhas and N. Ansari. 2012. Neural Networks in Telecommunications. Springer.

Digital Library

[76]

Y. Zhu, C.-H. Chan, U.-F. Chio, S.-W. Sin, U. Seng-Pan, R. P. Martins, and F. Aloberti. 2010. A 10-bit 100-MS/s reference-free SAR ADC in 90 nm CMOS. IEEE J. Solid-State Circ. 45, 1111--1121.

[77]

Q. Zou, Y. Bornat, J. Tomas, S. Renaud, and A. Destexhe. 2006. Real-time simulations of networks of Hodgkin--Huxley neurons using analog circuits. Neurocomput. 69, 1137--1140.

Digital Library

Cited By

Zheng H(Cindy) Yi Y(2023)Spiking Neural Encoding and Hardware Implementations for Neuromorphic ComputingNeuromorphic Computing10.5772/intechopen.113050Online publication date: 15-Nov-2023
https://doi.org/10.5772/intechopen.113050
Nowshin FAn HYi Y(2023)Towards Energy-Efficient Spiking Neural Networks: A Robust Hybrid CMOS-Memristive AcceleratorACM Journal on Emerging Technologies in Computing Systems10.1145/363516520:1(1-20)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3635165
Zheng HYi YSha KBanerjee SChen J(2023)Spiking Neural Encoding Schemes and STDP Training Algorithms for Edge ComputingProceedings of the Eighth ACM/IEEE Symposium on Edge Computing10.1145/3583740.3626816(365-371)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3583740.3626816
Show More Cited By

Index Terms

Spike-Time-Dependent Encoding for Neuromorphic Processors
1. Hardware
  1. Emerging technologies

Recommendations

Effects of Noise on Leaky Integrate-and-Fire Neuron Models for Neuromorphic Computing Applications
Computational Science and Its Applications – ICCSA 2022
Abstract
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), ...
Population encoding with Hodgkin-Huxley neurons
Special issue on information theory in molecular biology and neuroscience

The recovery of (weak) stimuli encoded with a population of Hodgkin-Huxley neurons is investigated. In the absence of a stimulus, the Hodgkin-Huxley neurons are assumed to be tonically spiking. The methodology employed calls for 1) finding an input-...
Interplay of the magnitude and time-course of postsynaptic Ca²⁺ concentration in producing spike timing-dependent plasticity

Synaptic strength can be modified by the relative timing of pre- and postsynaptic activity, a phenomenon termed spike timing-dependent plasticity (STDP). Studies of neurons in the hippocampus and in other regions have found that when presynaptic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Journal on Emerging Technologies in Computing Systems

ACM Journal on Emerging Technologies in Computing Systems Volume 12, Issue 3

Special Issue on Cross-Layer System Design and Regular Papers

September 2015

207 pages

ISSN:1550-4832

EISSN:1550-4840

DOI:10.1145/2828988

Editor:
Krishnendu Chakrabarty
Duke University, USA

Issue’s Table of Contents

Copyright © 2015 ACM.

© 2015 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 21 September 2015

Accepted: 01 February 2015

Revised: 01 December 2014

Received: 01 July 2014

Published in JETC Volume 12, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

AFRL, under AFRL

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
835
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)5

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zheng H(Cindy) Yi Y(2023)Spiking Neural Encoding and Hardware Implementations for Neuromorphic ComputingNeuromorphic Computing10.5772/intechopen.113050Online publication date: 15-Nov-2023
https://doi.org/10.5772/intechopen.113050
Nowshin FAn HYi Y(2023)Towards Energy-Efficient Spiking Neural Networks: A Robust Hybrid CMOS-Memristive AcceleratorACM Journal on Emerging Technologies in Computing Systems10.1145/363516520:1(1-20)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3635165
Zheng HYi YSha KBanerjee SChen J(2023)Spiking Neural Encoding Schemes and STDP Training Algorithms for Edge ComputingProceedings of the Eighth ACM/IEEE Symposium on Edge Computing10.1145/3583740.3626816(365-371)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3583740.3626816
Zheng HBai KYi Y(2023)Enabling a New Methodology of Neural Coding: Multiplexing Temporal Encoding in Neuromorphic ComputingIEEE Transactions on Very Large Scale Integration (VLSI) Systems10.1109/TVLSI.2023.323451431:3(331-342)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TVLSI.2023.3234514
Liu SYi Y(2023)Knowledge Distillation between DNN and SNN for Intelligent Sensing Systems on Loihi Chip2023 24th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED57927.2023.10129306(1-8)Online publication date: 5-Apr-2023
https://doi.org/10.1109/ISQED57927.2023.10129306
Date PKulkarni SYoung ASchuman CPotok TVetter J(2023)Encoding integers and rationals on neuromorphic computers using virtual neuronScientific Reports10.1038/s41598-023-35005-x13:1Online publication date: 6-Jul-2023
https://doi.org/10.1038/s41598-023-35005-x
Nowshin FYi Y(2023)ReRAM-Based Neuromorphic ComputingFrontiers of Quality Electronic Design (QED)10.1007/978-3-031-16344-9_2(43-65)Online publication date: 12-Jan-2023
https://doi.org/10.1007/978-3-031-16344-9_2
Nowshin FYi Y(2022)Memristor-based Deep Spiking Neural Network with a Computing-In-Memory Architecture2022 23rd International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED54688.2022.9806206(1-6)Online publication date: 6-Apr-2022
https://doi.org/10.1109/ISQED54688.2022.9806206
Date PKulkarni SYoung ASchuman CPotok TVetter J(2022)Virtual Neuron: A Neuromorphic Approach for Encoding Numbers2022 IEEE International Conference on Rebooting Computing (ICRC)10.1109/ICRC57508.2022.00017(100-105)Online publication date: Dec-2022
https://doi.org/10.1109/ICRC57508.2022.00017
Chien YXiang HShi YDuong NLi SAng K(2022) A MoS 2 Hafnium Oxide Based Ferroelectric Encoder for Temporal‐Efficient Spiking Neural Network Advanced Materials10.1002/adma.20220494935:2Online publication date: 29-Nov-2022
https://doi.org/10.1002/adma.202204949
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents