research-article

A Compact Memristor-Based Dynamic Synapse for Spiking Neural Networks

Authors:

J. Joshua Yang,

Hai Helen LiAuthors Info & Claims

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Volume 36, Issue 8

Pages 1353 - 1366

https://doi.org/10.1109/TCAD.2016.2618866

Published: 01 August 2017 Publication History

Abstract

Recent advances in memristor technology lead to the feasibility of large-scale neuromorphic systems by leveraging the similarity between memristor devices and synapses. For instance, memristor cross-point arrays can realize dense synapse network among hundreds of neuron circuits, which is not affordable for traditional implementations. However, little progress was made in synapse designs that support both static and dynamic synaptic properties. In addition, many neuron circuits require signals in specific pulse shape, limiting the scale of system implementation. Last but not least, a bottom-up study starting from realistic memristor devices is still missing in the current research of memristor-based neuromorphic systems. Here, we propose a memristor-based dynamic (MD) synapse design with experiment-calibrated memristor models. The structure obtains both static and dynamic synaptic properties by using one memristor for weight storage and the other as a selector. We overcame the device nonlinearities and demonstrated spike-timing-based recall, weight tunability, and spike-timing-based learning functions on MD synapse. Furthermore, a temporal pattern learning application was investigated to evaluate the use of MD synapses in spiking neural networks, under both spike-timing-dependent plasticity and remote supervised method learning rules.

References

[1]

W. Maass, “Networks of spiking neurons: The third generation of neural network models,” Neural Netw., vol. 10, no. 9, pp. 1659–1671, 1997.

[2]

E. Chicca, F. Stefanini, C. Bartolozzi, and G. Indiveri, “Neuromorphic electronic circuits for building autonomous cognitive systems,” Proc. IEEE, vol. 102, no. 9, pp. 1367–1388, Sep. 2014.

[3]

P. Merollaet al., “A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm,” in Proc. IEEE Custom Integr. Circuits Conf. (CICC), San Jose, CA, USA, 2011, pp. 1–4.

[4]

J.-S. Seoet al., “A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons,” in Proc. IEEE Custom Integr. Circuits Conf. (CICC), San Jose, CA, USA, 2011, pp. 1–4.

[5]

D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, “The missing memristor found,” Nature, vol. 453, no. 7191, pp. 80–83, 2008.

[6]

G. S. Snider, “Spike-timing-dependent learning in memristive nanodevices,” in Proc. IEEE Int. Symp. Nanoscale Architect., Anaheim, CA, USA, 2008, pp. 85–92.

[7]

S. H. Joet al., “Nanoscale memristor device as synapse in neuromorphic systems,” Nano Lett., vol. 10, no. 4, pp. 1297–1301, 2010.

[8]

S. Ambrogio, S. Balatti, F. Nardi, S. Facchinetti, and D. Ielmini, “Spike-timing dependent plasticity in a transistor-selected resistive switching memory,” Nanotechnology, vol. 24, no. 38, 2013, Art. no.

[9]

A. Thomas, “Memristor-based neural networks,” J. Phys. D Appl. Phys., vol. 46, no. 9, 2013, Art. no.

[10]

T. Ohnoet al., “Short-term plasticity and long-term potentiation mimicked in single inorganic synapses,” Nature Mater., vol. 10, no. 8, pp. 591–595, 2011.

[11]

T. Chang, S.-H. Jo, and W. Lu, “Short-term memory to long-term memory transition in a nanoscale memristor,” ACS Nano., vol. 5, no. 9, pp. 7669–7676, 2011.

[12]

H. Kim, M. P. Sah, C. Yang, T. Roska, and L. O. Chua, “Neural synaptic weighting with a pulse-based memristor circuit,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 59, no. 1, pp. 148–158, Jan. 2012.

[13]

K.-H. Kimet al., “A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications,” Nano Lett., vol. 12, no. 1, pp. 389–395, 2012.

[14]

S. Yu, Y. Wu, R. Jeyasingh, D. Kuzum, and H.-S. P. Wong, “An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation,” IEEE Trans. Electron Devices, vol. 58, no. 8, pp. 2729–2737, Aug. 2011.

[15]

M. Jerryet al., “Phase transition oxide neuron for spiking neural networks,” in Proc. 74th Annu. Device Res. Conf. (DRC), Newark, DE, USA, 2016, pp. 1–2.

[16]

S. Yuet al., “A neuromorphic visual system using RRAM synaptic devices with sub-pJ energy and tolerance to variability: Experimental characterization and large-scale modeling,” in Proc. IEEE Int. Electron Devices Meeting (IEDM), San Francisco, CA, USA, 2012, pp. 10.4.1–10.4.4.

[17]

M. Hu, H. Li, Q. Wu, and G. S. Rose, “Hardware realization of BSB recall function using memristor crossbar arrays,” in Proc. 49th Annu. Design Autom. Conf., San Francisco, CA, USA, 2012, pp. 498–503.

[18]

M. Huet al., “Dot-product engine for neuromorphic computing: Programming 1t1m crossbar to accelerate matrix-vector multiplication,” in Proc. 53rd Annu. Design Autom. Conf., Austin, TX, USA, 2016, pp. 1–6.

[19]

T. Gokmen and Y. Vlasov, “Acceleration of deep neural network training with resistive cross-point devices,” arXiv:1603.07341, 2016.

[20]

F. Ponulak and A. Kasiński, “Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classification, and spike shifting,” Neural Comput., vol. 22, no. 2, pp. 467–510, 2010.

Digital Library

[21]

W. Maass and A. M. Zador, “Dynamic stochastic synapses as computational units,” Neural Comput., vol. 11, no. 4, pp. 903–917, 1999.

Digital Library

[22]

M. Tsodyks, K. Pawelzik, and H. Markram, “Neural networks with dynamic synapses,” Neural Comput., vol. 10, no. 4, pp. 821–835, 1998.

Digital Library

[23]

N. Caporale and Y. Dan, “Spike timing-dependent plasticity: A Hebbian learning rule,” Annu. Rev. Neurosci., vol. 31, pp. 25–46, Feb. 2008.

[24]

J. P. Strachanet al., “State dynamics and modeling of tantalum oxide memristors,” IEEE Trans. Electron Devices, vol. 60, no. 7, pp. 2194–2202, Jul. 2013.

[25]

T. Gollisch and M. Meister, “Rapid neural coding in the retina with relative spike latencies,” Science, vol. 319, no. 5866, pp. 1108–1111, 2008.

[26]

E. Nordlie, M.-O. Gewaltig, and H. E. Plesser, “Towards reproducible descriptions of neuronal network models,” PLoS Comput. Biol., vol. 5, no. 8, 2009, Art. no.

[27]

G.-Q. Bi and M.-M. Poo, “Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type,” J. Neurosci., vol. 18, no. 24, pp. 10464–10472, 1998.

[28]

B. Widrow and M. A. Lehr, “30 years of adaptive neural networks: Perceptron, Madaline, and backpropagation,” Proc. IEEE, vol. 78, no. 9, pp. 1415–1442, Sep. 1990.

[29]

C. Mead, “Neuromorphic electronic systems,” Proc. IEEE, vol. 78, no. 10, pp. 1629–1636, Oct. 1990.

[30]

C. Mead and M. Ismail, Analog VLSI Implementation of Neural Systems. Boston, MA, USA: Springer, 1989.

[31]

C. Bartolozzi and G. Indiveri, “Synaptic dynamics in analog VLSI,” Neural Comput., vol. 19, no. 10, pp. 2581–2603, 2007.

Digital Library

[32]

M. Huet al., “Memristor crossbar-based neuromorphic computing system: A case study,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 10, pp. 1864–1878, Oct. 2014.

[33]

T. V. Bliss and G. L. Collingridge, “A synaptic model of memory: Long-term potentiation in the hippocampus,” Nature, vol. 361, no. 6407, pp. 31–39, 1993.

[34]

W. Gerstner and W. M. Kistler, Spiking Neuron Models: Single Neurons, Populations, Plasticity. New York, NY, USA: Cambridge Univ. Press, 2002.

[35]

D. A. Buttset al., “Temporal precision in the neural code and the timescales of natural vision,” Nature, vol. 449, no. 7158, pp. 92–95, 2007.

[36]

P. Tiesinga, J.-M. Fellous, and T. J. Sejnowski, “Regulation of spike timing in visual cortical circuits,” Nature Rev. Neurosci., vol. 9, no. 2, pp. 97–107, 2008.

[37]

R. VanRullen, R. Guyonneau, and S. J. Thorpe, “Spike times make sense,” Trends Neurosci., vol. 28, no. 1, pp. 1–4, 2005.

[38]

S. M. Bohte, “The evidence for neural information processing with precise spike-times: A survey,” Natural Comput., vol. 3, no. 2, pp. 195–206, 2004.

Digital Library

[39]

M. Meister and M. J. Berry, II, “The neural code of the retina,” Neuron, vol. 22, no. 3, pp. 435–450, 1999.

[40]

J. Hu, H. Tang, K. C. Tan, H. Li, and L. Shi, “A spike-timing-based integrated model for pattern recognition,” Neural Comput., vol. 25, no. 2, pp. 450–472, 2013.

[41]

H. Z. Shouval and G. Kalantzis, “Stochastic properties of synaptic transmission affect the shape of spike time-dependent plasticity curves,” J. Neurophysiol., vol. 93, no. 2, pp. 1069–1073, 2005.

[42]

J. J. Yanget al., “High switching endurance in TaO_x memristive devices,” Appl. Phys. Lett., vol. 97, no. 23, 2010, Art. no.

[43]

A. C. Torrezan, J. P. Strachan, G. Medeiros-Ribeiro, and R. S. Williams, “Sub-nanosecond switching of a tantalum oxide memristor,” Nanotechnology, vol. 22, no. 48, 2011, Art. no.

[44]

F. Miaoet al., “Anatomy of a nanoscale conduction channel reveals the mechanism of a high-performance memristor,” Adv. Mater., vol. 23, no. 47, pp. 5633–5640, 2011.

[45]

A. Prakash, D. Deleruyelle, J. Song, M. Bocquet, and H. Hwang, “Resistance controllability and variability improvement in a TaO_x-based resistive memory for multilevel storage application,” Appl. Phys. Lett., vol. 106, no. 23, 2015, Art. no.

[46]

M. Hu, Y. Wang, Q. Qiu, Y. Chen, and H. Li, “The stochastic modeling of TiO₂ memristor and its usage in neuromorphic system design,” in Proc. 19th Asia South Pac. Design Autom. Conf. (ASP-DAC), Singapore, Jan. 2014, pp. 831–836.

[47]

S. Song, K. D. Miller, and L. F. Abbott, “Competitive Hebbian learning through spike-timing-dependent synaptic plasticity,” Nature Neurosci., vol. 3, no. 9, pp. 919–926, 2000.

[48]

A. Sengupta, K. Yogendra, D. Fan, and K. Roy, “Prospects of efficient neural computing with arrays of magneto-metallic neurons and synapses,” in Proc. 21st Asia South Pac. Design Autom. Conf. (ASP-DAC), 2016, pp. 115–120.

[49]

Y.-C. Chen, H. Li, W. Zhang, and R. E. Pino, “The 3-D stacking bipolar RRAM for high density,” IEEE Trans. Nanotechnol., vol. 11, no. 5, pp. 948–956, Sep. 2012.

Digital Library

Cited By

Rathi NChakraborty IKosta ASengupta AAnkit APanda PRoy K(2023)Exploring Neuromorphic Computing Based on Spiking Neural Networks: Algorithms to HardwareACM Computing Surveys10.1145/357115555:12(1-49)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3571155
Deng ZWang CLin HSun Y(2023)A Memristive Spiking Neural Network Circuit With Selective Supervised Attention AlgorithmIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.322889642:8(2604-2617)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3228896
Yang XYang HDoppa JPande PChakrabartys KLi H(2023)ESSENCE: Exploiting Structured Stochastic Gradient Pruning for Endurance-Aware ReRAM-Based In-Memory Training SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.321654642:7(2187-2199)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3216546
Show More Cited By

Index Terms

A Compact Memristor-Based Dynamic Synapse for Spiking Neural Networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Memristor-based synaptic plasticity and unsupervised learning of spiking neural networks
Abstract
Synaptic plasticity is studied herein using a voltage-driven memristor model. The bidirectional weight update technique is demonstrated, and significant synaptic features, including nonlinear and threshold-based learning and long-term potentiation ...
Evolving unipolar memristor spiking neural networks

Neuromorphic computing – brain-like computing in hardware – typically requires myriad complimentary metal oxide semiconductor spiking neurons interconnected by a dense mesh of nanoscale plastic synapses. Memristors are frequently cited as strong synapse ...
Improved Izhikevich neurons for spiking neural networks

Spiking neural networks constitute a modern neural network paradigm that overlaps machine learning and computational neurosciences. Spiking neural networks use neuron models that possess a great degree of biological realism. The most realistic model of ...

Comments

Information & Contributors

Information

Published In

0278-0070 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 August 2017

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rathi NChakraborty IKosta ASengupta AAnkit APanda PRoy K(2023)Exploring Neuromorphic Computing Based on Spiking Neural Networks: Algorithms to HardwareACM Computing Surveys10.1145/357115555:12(1-49)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3571155
Deng ZWang CLin HSun Y(2023)A Memristive Spiking Neural Network Circuit With Selective Supervised Attention AlgorithmIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.322889642:8(2604-2617)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3228896
Yang XYang HDoppa JPande PChakrabartys KLi H(2023)ESSENCE: Exploiting Structured Stochastic Gradient Pruning for Endurance-Aware ReRAM-Based In-Memory Training SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.321654642:7(2187-2199)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3216546
Ghosh MMondal PBorah SKumar S(2023)Resistorless Memristor Emulators: Floating and Grounded Using OTA and VDBA for High-Frequency ApplicationsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.318983742:3(978-986)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3189837
Zhang YHe GTang KLi YWang G(2023)GEM: A Generalized Memristor Device Modeling Framework Based on Neural Network for Transient Circuit SimulationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.318896142:3(834-846)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3188961
Sun JYang JLiu PWang Y(2023)Design of General Flux-Controlled and Charge-Controlled Memristor Emulators Based on Hyperbolic FunctionsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.318692842:3(956-967)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3186928
Lin HWang CXu CZhang XIu H(2023)A Memristive Synapse Control Method to Generate Diversified Multistructure Chaotic AttractorsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.318651642:3(942-955)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3186516
Rziga FMbarek KGhedira SBesbes K(2022)An optimization of a non-volatile latch using memristors for sequential circuit applicationsAnalog Integrated Circuits and Signal Processing10.1007/s10470-021-01863-6110:1(55-61)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s10470-021-01863-6
Adnan MSayyaparaju SBrown SShawkat MSchuman CRose G(2021)Design of a Robust Memristive Spiking Neuromorphic System with Unsupervised Learning in HardwareACM Journal on Emerging Technologies in Computing Systems10.1145/345121017:4(1-26)Online publication date: 30-Jun-2021
https://dl.acm.org/doi/10.1145/3451210
Mbarek KRziga FGhedira SBesbes K(2020)On the design and analysis of a compact array with 1T1R RRAM memory elementAnalog Integrated Circuits and Signal Processing10.1007/s10470-019-01488-w102:1(27-37)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1007/s10470-019-01488-w

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents