research-article

Automatic abstraction and fault tolerance in cortical microachitectures

Authors:

Mikko LipastiAuthors Info & Claims

ISCA '11: Proceedings of the 38th annual international symposium on Computer architecture

Pages 1 - 10

https://doi.org/10.1145/2000064.2000066

Published: 04 June 2011 Publication History

Abstract

Recent advances in the neuroscientific understanding of the brain are bringing about a tantalizing opportunity for building synthetic machines that perform computation in ways that differ radically from traditional Von Neumann machines. These brain-like architectures, which are premised on our understanding of how the human neocortex computes, are highly fault-tolerant, averaging results over large numbers of potentially faulty components, yet manage to solve very difficult problems more reliably than traditional algorithms. A key principle of operation for these architectures is that of automatic abstraction: independent features are extracted from highly disordered inputs and are used to create abstract invariant representations of the external entities. This feature extraction is applied hierarchically, leading to increasing levels of abstraction at higher levels in the hierarchy.

This paper describes and evaluates a biologically plausible computational model for this process, and highlights the inherent fault tolerance of the biologically-inspired algorithm. We introduce a stuck-at fault model for such cortical networks, and describe how this model maps to hardware faults that can occur on commodity GPGPU cores used to realize the model in software. We show experimentally that the model software implementation can intrinsically preserve its functionality in the presence of faulty hardware, without requiring any reprogramming or recompilation. This model is a first step towards developing a comprehensive and biologically plausible understanding of the computational algorithms and microarchitecture of computing systems that mimic the human cortex, and to applying them to the robust implementation of tasks on future computing systems built of faulty components.

Supplementary Material

JPG File (isca_1_1.jpg)

Download
14.36 KB

MP4 File (isca_1_1.mp4)

Download
129.58 MB

References

[1]

R. Ananthanarayanan and D. S. Modha. Anatomy of a cortical simulator. In Proceedings of Supercomputing 07, November 2007.

Digital Library

[2]

A. Bakhoda, G. Yuan, W. Fung, H. Wong, and T. Aamondt.Analyzing cuda workloads using a detailed gpu simulator. In Proceedings of IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 163--174, 2009.

[3]

H. Berry and O. Temam. Modeling self-developping biological neural network. Neurocomputing, 70(16--18):2723--2734, 2007.

Digital Library

[4]

R. E. Brown and P. M. Milner. The legacy of Donald O. Hebb: more than the Hebb synapse. Nat Rev Neurosci, 4(12):1013--1019, Dec 2003.

[5]

N. Corporation. CUDA Programming Guide. NVIDIA Corporation, 2701 San Toman Expressway, Santa Clara, CA 95050, USA, 2007.

[6]

Systems of neuromorphic adaptive plastic scalable electronics (synapse). http://www.darpa.mil/dso/solicitations/baa08-28.html.

[7]

A. Dehon. Nanowire-based programmable architectures. J. Emerg. Technol. Comput. Syst., 1(2):109--162, 2005.

Digital Library

[8]

M. Emmerson and R. Damper. Determining and improving the fault tolerance of of multilayer perceptrons in a pattern-recognition application, 1993.

[9]

J. Fieres, K. Meier, and J. Schemmel. A convolutional neural network tolerant of synaptic faults for low-power analog hardware. In ANNPR, pages 122--132, 2006.

Digital Library

[10]

S. Haeusler and W. Maass. A statistical analysis of information-processing properties of lamina-specific cortical microcircuit models. Cereb. Cortex, 17(1):149--162, Jan 2007.

[11]

A. Hashmi, H. Berry, O. Temam, and M. H. Lipasti. Leveraging progress in neurobiology for computing systems. In Proceedings of the Workshop on New Directions in Computer Architecture held in Conjunction with 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO-42), 2009.

[12]

A. Hashmi and M. Lipasti. Cortical columns: Building blocks for intelligent systems. In Proceedings of the Symposium Series on Computational Intelligence, pages 21--28, 2009.

[13]

A. Hashmi and M. Lipasti. Discovering cortical algorithms. In Proceedings of the International Conference on Neural Comp utation (ICNC), October 2010.

[14]

S. Haykin. Neural Networks: A Comprehensive Foundation (2nd Edition). Prentice Hall, 1998.

Digital Library

[15]

J. Hirsch and L. Martinez. Laminar processing in the visual cortical column. Current Opinion in Neurobiology, 16:377--384, 2006.

[16]

M. Holler, S. Tam, H. Castro, and R. Benson. An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses. In International Joint Conference on Neural Networks, IJCNN, volume 2, pages 191--196, Jun 1989.

[17]

D. Hubel and T. Wiesel. Receptive fields, binocular interactions and functional architecture in cat's visual cortex. Journal of Physiology, 160:106--154, 1962.

[18]

K. Ibata, Q. Sun, and G. Turrigiano. Rapid Synaptic Scaling Induced by Changes in Postsynaptic Firing. Neuron, 57(6):819--826, 2008.

[19]

M. H. Kalisman N, Silberberg G. The neocortical microcircuit as a tabula rasa. Proc. Natl. Acad. Sci. USA, 102, 880--885, 2005.

[20]

E. Kandel, J. Schwartz, and T. Jessell. Principles of Neural Science. McGraw-Hill, 4 edition, 2000.

[21]

G. Kreiman, C. Koch, and I. Fried. Category-specific visual responses of single neurons in the human medial temporal lobe. Nature Neuroscience, 3:946--953, 2000.

[22]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278--2324, November 1998.

[23]

A. Losonczy and J. Magee. Integrative properties of radial oblique dendrites in hippocampal ca1 pyramidal neurons. Neuron, 50:291--307, 2006.

[24]

A. Nere and M. Lipasti. Cortical architectures on a gpgpu. In GPGPU '10: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units, pages 12--18, New York, NY, USA, 2010. ACM.

Digital Library

[25]

J. T. Paz, S. Mahon, P. Tiret, S. Genet, B. Delord, and S. Charpier. Multiple forms of activity-dependent intrinsic plasticity in layer V cortical neurones in vivo. The Journal of Physiology, 587(13):3189--3205, 2009.

[26]

J. Peissig and M. Tarr. Visual object recognition: do we know more now than we did 20 years ago? Annu. Rev. Psychol., 58:75--96, 2007.

[27]

M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. Nature Neuroscience, 2:1019--1025, 1999.

[28]

D. Ringach. Haphazard wiring of simple receptive fields and orientation columns in visual cortex. J. Neurophysiol., 92(1):468--476, Jul 2004.

[29]

S. Ryoo, C. Rodrigues, S. Babhsorkhi, S. Stone, D. Kirk, and W. Hwu. Optimization principles and application performance evaluation of a multithreaded gpu using cuda. In Proceesings Symposium on principles and practices of parallel programming, SIGPLAN, pages 73--82, 2008.

Digital Library

[30]

A. Sandberg and N. Bostrom. Whole brain emulation: A roadmap. Technical Report 2008-3, Future of Humanity Institute, Oxford University, 2008.

[31]

J. Schemmel, J. Fieres, and K. Meier. Wafer-scale integration of analog neural networks. In IEEE International Joint Conference on Neural Networks, IJCNN, pages 431--438, June 2008.

[32]

T. Serre, A. Oliva, and T. Poggio. A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. USA, 104(15):6424--6429, Apr 2007.

[33]

T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, and T. Poggio. Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intell., 29(3):411--426, Mar 2007.

Digital Library

[34]

SIA. Semiconductor industry association 2007 roadmap. http://www.sia-online.org/, 2007.

[35]

G. Sperling. A model for visual memory tasks. Human Factor, 5:19--31, 1963.

[36]

M. Spratling. Presynaptic lateral inhibition provides a betterarchitecture for self-organising neural networks. Network: Computation in Neural Systems, 10(4):285--301, 1999.

[37]

R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.

Digital Library

[38]

L. Swanson. Mapping the human brain: past, present, and future. Trends in Neurosciences, 18(11):471--474, 1995.

Cited By

Venkatesha SParthasarathi R(2024)Survey on Redundancy Based-Fault tolerance methods for Processors and Hardware accelerators - Trends in Quantum Computing, Heterogeneous Systems and ReliabilityACM Computing Surveys10.1145/366367256:11(1-76)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3663672
Su FLiu CStratigopoulos H(2023)Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and PerspectivesIEEE Design & Test10.1109/MDAT.2023.324111640:2(8-58)Online publication date: Apr-2023
https://doi.org/10.1109/MDAT.2023.3241116
Spyrou TStratigopoulos H(2023)On-Line Testing of Neuromorphic Hardware2023 IEEE European Test Symposium (ETS)10.1109/ETS56758.2023.10174077(1-6)Online publication date: 22-May-2023
https://doi.org/10.1109/ETS56758.2023.10174077
Show More Cited By

Index Terms

Automatic abstraction and fault tolerance in cortical microachitectures
1. Computer systems organization
  1. Architectures
    1. Other architectures
2. Hardware
  1. Hardware test
  2. Robustness

Recommendations

Automatic abstraction and fault tolerance in cortical microachitectures
ISCA '11

Recent advances in the neuroscientific understanding of the brain are bringing about a tantalizing opportunity for building synthetic machines that perform computation in ways that differ radically from traditional Von Neumann machines. These brain-like ...
Fault Tolerance in Multiprocessor Systems Without Dedicated Redundancy

An algorithm called RAFT (recursive algorithm for fault tolerance) for achieving fault tolerance in multiprocessor systems is described. Through the use of a combination of dynamic space- and time- redundancy techniques, RAFT achieves fault tolerance in ...
Using dynamic task level redundancy for OpenMP fault tolerance
ARCS'12: Proceedings of the 25th international conference on Architecture of Computing Systems

Obtaining fault tolerant applications and systems is one of today's most important topics of research. Fault tolerance is becoming more and more essential in shared memory parallel programs and in multi/many core architectures due to the decreasing size ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISCA '11: Proceedings of the 38th annual international symposium on Computer architecture

June 2011

488 pages

ISBN:9781450304726

DOI:10.1145/2000064

General Chairs:
Ravi Iyer
Intel
,
Qing Yang
University of Rhode Island
,
Program Chair:
Antonio González
Intel and UPC

ACM SIGARCH Computer Architecture News Volume 39, Issue 3
ISCA '11
June 2011
462 pages
ISSN:0163-5964
DOI:10.1145/2024723
Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISCA '11

Sponsor:

SIGARCH

ISCA '11: The 38th Annual International Symposium on Computer Architecture

June 4 - 8, 2011

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 543 of 3,203 submissions, 17%

Upcoming Conference

ISCA '25

Sponsor:
sigarch

The 52nd Annual International Symposium on Computer Architecture

June 21 - 25, 2025

Tokyo , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
870
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)4

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Venkatesha SParthasarathi R(2024)Survey on Redundancy Based-Fault tolerance methods for Processors and Hardware accelerators - Trends in Quantum Computing, Heterogeneous Systems and ReliabilityACM Computing Surveys10.1145/366367256:11(1-76)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3663672
Su FLiu CStratigopoulos H(2023)Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and PerspectivesIEEE Design & Test10.1109/MDAT.2023.324111640:2(8-58)Online publication date: Apr-2023
https://doi.org/10.1109/MDAT.2023.3241116
Spyrou TStratigopoulos H(2023)On-Line Testing of Neuromorphic Hardware2023 IEEE European Test Symposium (ETS)10.1109/ETS56758.2023.10174077(1-6)Online publication date: 22-May-2023
https://doi.org/10.1109/ETS56758.2023.10174077
Stratigopoulos HSpyrou TRaptis S(2023)Testing and Reliability of Spiking Neural Networks: A Review of the State-of-the-Art2023 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT)10.1109/DFT59622.2023.10313541(1-8)Online publication date: 3-Oct-2023
https://doi.org/10.1109/DFT59622.2023.10313541
Lilak SWoods WScharnhorst KDunham CTeuscher CStieg AGimzewski J(2021)Spoken Digit Classification by In-Materio Reservoir Computing With Neuromorphic Atomic Switch NetworksFrontiers in Nanotechnology10.3389/fnano.2021.6757923Online publication date: 26-May-2021
https://doi.org/10.3389/fnano.2021.675792
Kang KPark JLee Y(2019)Development of Vital Sensing Transmission System based on Internet of Thing (IoT) sensorJournal of Digital Contents Society10.9728/dcs.2019.20.2.22120:2(221-229)Online publication date: 28-Feb-2019
https://doi.org/10.9728/dcs.2019.20.2.221
Hajj NAwad M(2019)A piecewise weight update rule for a supervised training of cortical algorithmsNeural Computing and Applications10.1007/s00521-017-3167-531:6(1915-1930)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s00521-017-3167-5
Liu JHarkin JMaguire LMcDaid LWade J(2018)SPANNER: A Self-Repairing Spiking Neural Network Hardware ArchitectureIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2017.267302129:4(1287-1300)Online publication date: Apr-2018
https://doi.org/10.1109/TNNLS.2017.2673021
Riera MArnau JGonzález A(2018)Computation reuse in DNNs by exploiting input similarityProceedings of the 45th Annual International Symposium on Computer Architecture10.1109/ISCA.2018.00016(57-68)Online publication date: 2-Jun-2018
https://dl.acm.org/doi/10.1109/ISCA.2018.00016
Bhattacharjee AHunter HMoreno JEmer JSanchez D(2017)Using branch predictors to predict brain activity in brain-machine implantsProceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3123939.3123943(409-422)Online publication date: 14-Oct-2017
https://dl.acm.org/doi/10.1145/3123939.3123943
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents