research-article

Open access

A System-Level Simulator for RRAM-Based Neuromorphic Computing Chips

Authors:

Matthew Kay Fei Lee,

Thannirmalai Somu,

Weng-Fai Wong, and

Rick Siow Mong GohAuthors Info & Claims

ACM Transactions on Architecture and Code Optimization (TACO), Volume 15, Issue 4

Article No.: 64, Pages 1 - 24

https://doi.org/10.1145/3291054

Published: 08 January 2019 Publication History

All formats PDF

Abstract

Advances in non-volatile resistive switching random access memory (RRAM) have made it a promising memory technology with potential applications in low-power and embedded in-memory computing devices owing to a number of advantages such as low-energy consumption, low area cost and good scaling. There have been proposals to employ RRAM in architecting chips for neuromorphic computing and artificial neural networks where matrix-vector multiplication can be computed in the analog domain in a single timestep. However, it is challenging to employ RRAM devices in neuromorphic chips owing to the non-ideal behavior of RRAM. In this article, we propose a cycle-accurate and scalable system-level simulator that can be used to study the effects of using RRAM devices in neuromorphic computing chips. The simulator models a spatial neuromorphic chip architecture containing many neural cores with RRAM crossbars connected via a Network-on-Chip (NoC). We focus on system-level simulation and demonstrate the effectiveness of our simulator in understanding how non-linear RRAM effects such as stuck-at-faults (SAFs), write variability, and random telegraph noise (RTN) can impact an application’s behavior. By using our simulator, we show that RTN and write variability can have adverse effects on an application. Nevertheless, we show that these effects can be mitigated through proper design choices and the implementation of a write-verify scheme.

References

[1]

Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. 2015. TrueNorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 34, 10 (2015), 1537--1557.

Digital Library

[2]

J. Arthur and K. Boahen. 2004. Recurrently connected silicon neurons with active dendrites for one-shot learning. In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN’04), Vol. 3. IEEE, 1699--1704.

[3]

Bhaswar Chakrabarti, Miguel Angel Lastras-Montaño, Gina Adam, Mirko Prezioso, Brian Hoskins, M. Payvand, A. Madhavan, A. Ghofrani, L. Theogarajan, K.-T. Cheng, et al. 2017. A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit. Scientific Reports 7 (2017), 42429.

[4]

A. Ankit et al. 2017. RESPARC: A reconfigurable and energy-efficient architecture with memristive crossbars for deep spiking neural networks. In Proceedings of the 54th Annual Design Automation Conference 2017. ACM, 27:1--27:6.

Digital Library

[5]

A. Davison et al. 2009. PyNN: A common interface for neuronal network simulators. Frontiers in Neuroinformatics 2 (2009), 11.

[6]

A. Shafiee et al. 2016. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA). 14--26.

Digital Library

[7]

B. Benjamin et al. 2014. Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proceedings of the IEEE 102, 5 (2014), 699--716.

[8]

D. Yudanov et al. 2010. GPU-based simulation of spiking neural networks with real-time performance and high accuracy. In International Joint Conference on Neural Networks (IJCNN’10). IEEE, 1--8.

[9]

E. Painkras et al. 2013. SpiNNaker: A 1-W 18-core system-on-chip for massively-parallel neural network simulation. IEEE Journal of Solid-State Circuits 48, 8 (Aug. 2013), 1943--1953.

[10]

G. Indiveri et al. 2013. Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology 24, 38 (2013), 384010.

[11]

H. Wong et al. 2012. Metal-oxide RRAM. Proceedings of the IEEE 100, 6 (June 2012), 1951--1970.

[12]

J. Schemmel et al. 2010. A wafer-scale neuromorphic hardware system for large-scale neural modeling. In Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS’10). IEEE, 1947--1950.

[13]

K. Minkovich et al. 2014. HRLSim: A high performance spiking neural network simulator for GPGPU clusters. IEEE Transactions on Neural Networks and Learning Systems 25, 2 (2014), 316--331.

[14]

L. Xia et al. 2018. MNSIM: Simulation platform for memristor-based neuromorphic computing system. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37, 5 (May 2018), 1009--1022.

[15]

M. Hu et al. 2016. Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication. In 53rd ACM/EDAC/IEEE Design Automation Conference (DAC’16). 1--6.

Digital Library

[16]

M. Prezioso et al. 2015. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 7550 (2015), 61--64.

[17]

P. Merolla et al. 2014. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 6197 (2014), 668--673.

[18]

P. U. Diehl et al. 2015. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In International Joint Conference on Neural Networks (IJCNN’15). 1--8.

[19]

P. Yao et al. 2017. Face classification using electronic synapses. Nature Communications 8 (2017), 15199.

[20]

S. Ambrogio et al. 2014. Statistical fluctuations in HfOx resistive-switching memory: Part I - Set/reset variability. IEEE Transactions on Electron Devices 61, 8 (Aug 2014), 2912--2919.

[21]

S. Ambrogio et al. 2014. Statistical fluctuations in HfOx resistive-switching memory: Part II - Random telegraph noise. IEEE Transactions on Electron Devices 61, 8 (Aug 2014), 2920--2927.

[22]

S. Esser et al. 2016. Convolutional networks for fast, energy-efficient neuromorphic computing. Proceedings of the National Academy of Sciences 113, 41 (2016), 11441--11446.

[23]

S. H. Jo et al. 2010. Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters 10, 4 (2010), 1297--1301.

[24]

S. Narayanan et al. 2017. INXS: Bridging the throughput and energy gap for spiking neural networks. In International Joint Conference on Neural Networks (IJCNN’17). IEEE, 2451--2459.

[25]

S. Park et al. 2013. Neuromorphic speech systems using advanced ReRAM-based synapse. In IEEE International Electron Devices Meeting (IEDM’13). IEEE, 25--6.

[26]

T. Tang et al. 2015. Spiking neural network with RRAM: Can we use it for real-world application? In Proceedings of the Design, Automation 8 Test in Europe Conference 8 Exhibition. EDA Consortium, 860--865.

Digital Library

[27]

A. Fidjeland and M. Shanahan. 2010. Accelerated simulation of spiking neural networks using GPUs. In International Joint Conference on Neural Networks (IJCNN’10). IEEE, 1--8.

[28]

M. Gewaltig and M. Diesmann. 2007. NEST (NEural simulation tool). Scholarpedia 2, 4 (2007), 1430.

[29]

Amirali Ghofrani, Siddharth Gaba, Melika Payvand, Wei Lu, Luke Theogarajan, Kwang-Ting Cheng, et al. 2015. A low-power variation-aware adaptive write scheme for access-transistor-free memristive memory. ACM Journal on Emerging Technologies in Computing Systems (JETC) 12, 1 (2015), 3.

Digital Library

[30]

D. Goodman and R. Brette. 2009. The brain simulator. Frontiers in Neuroscience 3 (2009), 26.

[31]

Miao Hu, John Paul Strachan, Zhiyong Li, R. Stanley, et al. 2016. Dot-product engine as computing memory to accelerate machine learning algorithms. In 17th International Symposium on Quality Electronic Design (ISQED’16). IEEE, 374--379.

[32]

E. M. Izhikevich. 2003. Simple model of spiking neurons. IEEE Transactions on Neural Networks 14, 6 (Nov 2003), 1569--1572.

Digital Library

[33]

Yu Ji, Youhui Zhang, Wenguang Chen, and Yuan Xie. 2018. Bridge the gap between neural networks and neuromorphic hardware with a neural network compiler. In Proceedings of the 23rd International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, 448--460.

Digital Library

[34]

Yu Ji, Youhui Zhang, He Liu, and Weimin Zheng. 2016. Optimized mapping spiking neural networks onto network-on-chip. In International Conference on Algorithms and Architectures for Parallel Processing. Springer, 38--52.

[35]

Andrew B. Kahng, Bin Li, Li-Shiuan Peh, and Kambiz Samadi. 2009. ORION 2.0: A fast and accurate NoC power and area model for early-stage design space exploration. In Proceedings of the Conference on Design, Automation and Test in Europe. European Design and Automation Association, 423--428.

Digital Library

[36]

V. G. Karpov and D. Niraula. 2017. Log-normal statistics in filamentary RRAM devices and related systems. IEEE Electron Device Letters 38, 9 (2017), 1240--1243.

[37]

Boxun Li, Lixue Xia, Peng Gu, Yu Wang, and Huazhong Yang. 2015. Merging the interface: Power, area and accuracy co-optimization for RRAM crossbar-based mixed-signal computing system. In Proceedings of the 52nd Annual Design Automation Conference. ACM, 13.

Digital Library

[38]

Ye Li, Bertan Bakkaloglu, and Chaitali Chakrabarti. 2007. A system level energy model and energy-quality evaluation for integrated transceiver front-ends. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 15, 1 (2007), 90--103.

Digital Library

[39]

W. Maass. 1997. Networks of spiking neurons: The third generation of neural network models. Neural Networks 10, 9 (1997), 1659--1671.

[40]

Don Monroe. 2014. Neuromorphic computing gets ready for the (really) big time. Communications of the ACM 57, 6 (2014), 13--15.

Digital Library

[41]

Hiroki Noguchi, Kazutaka Ikegami, Satoshi Takaya, Eishi Arima, Keiichi Kushida, Atsushi Kawasumi, Hiroyuki Hara, Keiko Abe, Naoharu Shimomura, Junichi Ito, et al. 2016. 7.2 4MB STT-MRAM-based cache with memory-access-aware power optimization and write-verify-write/read-modify-write scheme. In IEEE International Solid-State Circuits Conference (ISSCC’16). IEEE, 132--133.

[42]

M. Papamichael and J. Hoe. 2012. CONNECT: Re-examining conventional wisdom for designing NoCs in the context of FPGAs. In Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays. 37--46.

Digital Library

[43]

R. Preisslet al. 2012. COMPASS: A scalable simulator for an architecture for cognitive computing. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. 54.

Digital Library

[44]

D. Querlioz, O. Bichler, and C. Gamrat. 2011. Simulation of a memristor-based spiking neural network immune to device variations. In International Joint Conference on Neural Networks (IJCNN’11). IEEE, 1775--1781.

[45]

K. Zaghloul and K. Boahen. 2004. Optic nerve signals in a neuromorphic chip I: Outer and inner retina models. IEEE Transactions on Biomedical Engineering 51, 4 (2004), 657--666.

Cited By

Kim MKim S(2024)From Oxides to 2D Materials: Advancing Memristor Technologies for Energy-Efficient Neuromorphic ComputingACS Applied Electronic Materials10.1021/acsaelm.4c004286:6(3998-4015)Online publication date: 30-May-2024
https://doi.org/10.1021/acsaelm.4c00428
Zhang HHo NPolat DChen PWahib MNguyen TMeng JGoh RMatsuoka SLuo TWong W(2023)Simeuro: A Hybrid CPU-GPU Parallel Simulator for Neuromorphic Computing ChipsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.329179534:10(2767-2782)Online publication date: Oct-2023
https://doi.org/10.1109/TPDS.2023.3291795
Wei MYayla MHo SChen JAmrouch HYang C(2023)Impact of Non-Volatile Memory Cells on Spiking Neural Network Annealing Machine With In-Situ Synapse ProcessingIEEE Transactions on Circuits and Systems I: Regular Papers10.1109/TCSI.2023.330501070:11(4380-4393)Online publication date: Nov-2023
https://doi.org/10.1109/TCSI.2023.3305010
Show More Cited By

Index Terms

A System-Level Simulator for RRAM-Based Neuromorphic Computing Chips

Recommendations

Research on feasibility of using a Transient Voltage Suppressor as the selection device for bipolar RRAM

In this paper, the Transient Voltage Suppressor (TVS) is provided as the selection device for bipolar RRAM crossbar array applications. Simulation results suggest that, by using the TVS as a selection device, the sneak current in the crossbar array can ...
Read More
Oxide based two diodes-one resistor structure for bipolar RRAM crossbar array

Graphical abstractI-V characteristics of the anti-parallel connected Ag/TiOx/Ti diodes selector in series with the Cu/HfO2/Pt bipolar RRAM cell.Display Omitted A new bipolar selector based on anti-parallel connected diodes is proposed.Anti-parallel ...
Read More
An RRAM-based MLC design approach

RRAM is an emerging technology with vast applications in computation and storage. RRAM with MLC capabilities is more interesting owing to its higher storage density. In this study, we proposed an approach for RRAM-based MLC design. The approach starts ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Architecture and Code Optimization

ACM Transactions on Architecture and Code Optimization Volume 15, Issue 4

December 2018

706 pages

ISSN:1544-3566

EISSN:1544-3973

DOI:10.1145/3284745

Editor:
Koen De Bosschere
Ghent University

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 January 2019

Accepted: 01 October 2018

Revised: 01 September 2018

Received: 01 May 2018

Published in TACO Volume 15, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

programmatic
Singapore Government Research, Innovation and Enterprise 2020 plan
Advanced Manufacturing and Engineering domain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
3,981
Total Downloads

Downloads (Last 12 months)609
Downloads (Last 6 weeks)42

Other Metrics

View Author Metrics

Citations

Cited By

Kim MKim S(2024)From Oxides to 2D Materials: Advancing Memristor Technologies for Energy-Efficient Neuromorphic ComputingACS Applied Electronic Materials10.1021/acsaelm.4c004286:6(3998-4015)Online publication date: 30-May-2024
https://doi.org/10.1021/acsaelm.4c00428
Zhang HHo NPolat DChen PWahib MNguyen TMeng JGoh RMatsuoka SLuo TWong W(2023)Simeuro: A Hybrid CPU-GPU Parallel Simulator for Neuromorphic Computing ChipsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.329179534:10(2767-2782)Online publication date: Oct-2023
https://doi.org/10.1109/TPDS.2023.3291795
Wei MYayla MHo SChen JAmrouch HYang C(2023)Impact of Non-Volatile Memory Cells on Spiking Neural Network Annealing Machine With In-Situ Synapse ProcessingIEEE Transactions on Circuits and Systems I: Regular Papers10.1109/TCSI.2023.330501070:11(4380-4393)Online publication date: Nov-2023
https://doi.org/10.1109/TCSI.2023.3305010
Park JShin YSung H(2023)Multi-Objective Architecture Search and Optimization for Heterogeneous Neuromorphic Architecture2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)10.1109/ICCAD57390.2023.10323779(1-8)Online publication date: 28-Oct-2023
https://doi.org/10.1109/ICCAD57390.2023.10323779
Xie LXue JWu LChen FTian QZhou YYing RLiu P(2023)SpikeNC: An Accurate and Scalable Simulator for Spiking Neural Network on Multi-Core Neuromorphic Hardware2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC58850.2023.00052(357-366)Online publication date: 18-Dec-2023
https://doi.org/10.1109/HiPC58850.2023.00052
Zhang JZhang JHuo DChen H(2023)ANAS: Asynchronous Neuromorphic Hardware Architecture Search Based on a System-Level Simulator2023 60th ACM/IEEE Design Automation Conference (DAC)10.1109/DAC56929.2023.10247850(1-6)Online publication date: 9-Jul-2023
https://doi.org/10.1109/DAC56929.2023.10247850
Song MKang JZhang XJi WAscoli AMessaris IDemirkol ADong BAggarwal SWan WHong SCardwell SBoybat ISeo JLee JLanza MYeon HOnen MLi JYildiz Bdel Alamo JKim SChoi SMilano GRicciardi CAlff LChai YWang ZBhaskaran HHersam MStrukov DWong HValov IGao BWu HTetzlaff RSebastian ALu WChua LYang JKim J(2023)Recent Advances and Future Prospects for Memristive Materials, Devices, and SystemsACS Nano10.1021/acsnano.3c0350517:13(11994-12039)Online publication date: 29-Jun-2023
https://doi.org/10.1021/acsnano.3c03505
Yang KYin ZWu JDeng HYang ZFan ZZhao J(2023)Ultra-low power resistive random-access memory based on VO2/TiO2 nanotubes composite filmVacuum10.1016/j.vacuum.2023.112472216(112472)Online publication date: Oct-2023
https://doi.org/10.1016/j.vacuum.2023.112472
Mysore NHota GDeiss SPedroni BCauwenberghs G(2022)Hierarchical Network Connectivity and Partitioning for Reconfigurable Large-Scale Neuromorphic SystemsFrontiers in Neuroscience10.3389/fnins.2021.79765415Online publication date: 31-Jan-2022
https://doi.org/10.3389/fnins.2021.797654
Zahedi MLebdeh MBengel CWouters DMenzel SLe Gallo MSebastian AWong SHamdioui S(2022)MNEMOSENE: Tile Architecture and Simulator for Memristor-based Computation-in-memoryACM Journal on Emerging Technologies in Computing Systems10.1145/348582418:3(1-24)Online publication date: 29-Jan-2022
https://dl.acm.org/doi/10.1145/3485824
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents