research-article

Exploring Information-Theoretic Criteria to Accelerate the Tuning of Neuromorphic Level-Crossing ADCs

Authors:

Jonah Van Assche,

Charlotte Frenkel,

Andre Bourdoux,

Francky Catthoor,

Georges GielenAuthors Info & Claims

NICE '23: Proceedings of the 2023 Annual Neuro-Inspired Computational Elements Conference

Pages 63 - 70

https://doi.org/10.1145/3584954.3584994

Published: 12 April 2023 Publication History

Abstract

Level-crossing analog-to-digital converters (LC-ADCs) are neuromorphic, event-driven data converters that are gaining much attention for resource-constrained applications where intelligent sensing must be provided at the extreme edge, with tight energy and area budgets. LC-ADCs translate real-world analog signals (such as ECG, EEG, etc.) into sparse spiking signals, providing significant data bandwidth reduction and inducing savings of up to two orders of magnitude in area and energy consumption at the system level compared to the use of conventional ADCs. In addition, the spiking nature of LC-ADCs make their use a natural choice for ultra-low-power, event-driven spiking neural networks (SNNs). Still, the compressed nature of LC-ADC spiking signals can jeopardize the performance of downstream tasks such as signal classification accuracy, which is highly sensitive to the LC-ADC tuning parameters. In this paper, we explore the use of popular information criteria found in model selection theory for the tuning of the LC-ADC parameters. We experimentally demonstrate that information metrics such as the Bayesian, Akaike and corrected Akaike criteria can be used to tune the LC-ADC parameters in order to maximize downstream SNN classification accuracy. We conduct our experiments using both full-resolution weights and 4-bit quantized SNNs, on two different bio-signal classification tasks. We believe that our findings can accelerate the tuning of LC-ADC parameters without resorting to computationally-expensive grid searches that require many SNN training passes.

References

[1]

H. Akaike. 1974. A new look at the statistical model identification. IEEE Trans. Automat. Control 19, 6 (1974), 716–723. https://doi.org/10.1109/TAC.1974.1100705

[2]

Mark Daniel Alea, Ali Safa, Jonah Van Assche, and Georges G. E. Gielen. 2022. Power-Efficient and Accurate Texture Sensing Using Spiking Readouts for High-Density e-Skins. In 2022 IEEE Biomedical Circuits and Systems Conference (BioCAS). 359–363. https://doi.org/10.1109/BioCAS54905.2022.9948546

[3]

Ralph Andrzejak, Klaus Lehnertz, Florian Mormann, Christoph Rieke, Peter David, and Christian Elger. 2002. Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Physical review. E, Statistical, nonlinear, and soft matter physics 64 (01 2002), 061907. https://doi.org/10.1103/PhysRevE.64.061907

[4]

K.P. Burnham and D.R. Anderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach. Springer Verlag.

[5]

Joseph E. Cavanaugh. 1997. Unifying the derivations for the Akaike and corrected Akaike information criteria. Statistics & Probability Letters 33, 2 (30 April 1997), 201–208. http://www.sciencedirect.com/science/article/B6V1D-3WNMX5B-X/1/fcad33e5e0d138a917cb28ab3e5b2d43

[6]

Charlotte Frenkel, Martin Lefebvre, Jean-Didier Legat, and David Bol. 2019. A 0.086-mm2 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS. IEEE Transactions on Biomedical Circuits and Systems 13, 1 (2019), 145–158. https://doi.org/10.1109/TBCAS.2018.2880425

[7]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 9), Yee Whye Teh and Mike Titterington (Eds.). PMLR, Chia Laguna Resort, Sardinia, Italy, 249–256. https://proceedings.mlr.press/v9/glorot10a.html

[8]

Yuming He, Federico Corradi, Chengyao Shi, Ming Ding, Martijn Timmermans, Jan Stuijt, Pieter Harpe, Ilja Ocket, and Yao-Hong Liu. 2021. A 28.2 μC Neuromorphic Sensing System Featuring SNN-based Near-sensor Computation and Event-Driven Body-Channel Communication for Insertable Cardiac Monitoring. In 2021 IEEE Asian Solid-State Circuits Conference (A-SSCC). 1–3. https://doi.org/10.1109/A-SSCC53895.2021.9634787

[9]

Yuming He, Federico Corradi, Chengyao Shi, Stan van der Ven, Martijn Timmermans, Jan Stuijt, Paul Detterer, Pieter Harpe, Lucas Lindeboom, Evelien Hermeling, Geert Langereis, Elisabetta Chicca, and Yao-Hong Liu. 2022. An Implantable Neuromorphic Sensing System Featuring Near-Sensor Computation and Send-on-Delta Transmission for Wireless Neural Sensing of Peripheral Nerves. IEEE Journal of Solid-State Circuits 57, 10 (2022), 3058–3070. https://doi.org/10.1109/JSSC.2022.3193846

[10]

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. https://doi.org/10.48550/ARXIV.1412.6980

[12]

Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. 2008. A 128 × 128 120 dB 15 μs Latency Asynchronous Temporal Contrast Vision Sensor. IEEE Journal of Solid-State Circuits 43, 2 (2008), 566–576. https://doi.org/10.1109/JSSC.2007.914337

[13]

G.B. Moody and R.G. Mark. 1990. The MIT-BIH Arrhythmia Database on CD-ROM and software for use with it. In [1990] Proceedings Computers in Cardiology. 185–188. https://doi.org/10.1109/CIC.1990.144205

[14]

Simon F. Müller-Cleve, Vittorio Fra, Lyes Khacef, Alejandro Pequeño-Zurro, Daniel Klepatsch, Evelina Forno, Diego G. Ivanovich, Shavika Rastogi, Gianvito Urgese, Friedemann Zenke, and Chiara Bartolozzi. 2022. Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware. Frontiers in Neuroscience 16 (2022). https://doi.org/10.3389/fnins.2022.951164

[15]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, Oct (2011), 2825–2830.

Digital Library

[16]

Balint Petro, Nikola Kasabov, and Rita M. Kiss. 2020. Selection and Optimization of Temporal Spike Encoding Methods for Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems 31, 2(2020), 358–370. https://doi.org/10.1109/TNNLS.2019.2906158

[17]

Ali Safa, Federico Corradi, Lars Keuninckx, Ilja Ocket, André Bourdoux, Francky Catthoor, and Georges G. E. Gielen. 2021. Improving the Accuracy of Spiking Neural Networks for Radar Gesture Recognition Through Preprocessing. IEEE Transactions on Neural Networks and Learning Systems (2021), 1–13. https://doi.org/10.1109/TNNLS.2021.3109958

[18]

Ali Safa, Jonah Van Assche, Mark Daniel Alea, Francky Catthoor, and Georges G.E. Gielen. 2022. Neuromorphic Near-Sensor Computing: From Event-Based Sensing to Edge Learning. IEEE Micro 42, 6 (2022), 88–95. https://doi.org/10.1109/MM.2022.3195634

Digital Library

[19]

Rohan Sanghavi, Fenil Chheda, Sachin Kanchan, and Sushma Kadge. 2021. Detection Of Atrial Fibrillation in Electrocardiogram Signals using Machine Learning. In 2021 2nd Global Conference for Advancement in Technology (GCAT). 1–6. https://doi.org/10.1109/GCAT52182.2021.9587664

[20]

B. Schrauwen and J. Van Campenhout. 2003. BSA, a fast and accurate spike train encoding scheme. In Proceedings of the International Joint Conference on Neural Networks, 2003., Vol. 4. 2825–2830 vol.4. https://doi.org/10.1109/IJCNN.2003.1224019

[21]

Sumit Bam Shrestha and Garrick Orchard. 2018. SLAYER: Spike Layer Error Reassignment in Time. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montréal, Canada) (NIPS’18). Curran Associates Inc., Red Hook, NY, USA, 1419–1428.

[22]

Jonah Van Assche and Georges Gielen. 2020. Power Efficiency Comparison of Event-Driven and Fixed-Rate Signal Conversion and Compression for Biomedical Applications. IEEE Transactions on Biomedical Circuits and Systems 14, 4 (2020), 746–756. https://doi.org/10.1109/TBCAS.2020.3009027

[23]

Jonah Van Assche and Georges Gielen. 2022. A 10.4-ENOB 0.92-5.38 μW Event-Driven Level-Crossing ADC with Adaptive Clocking for Time-Sparse Edge Applications. In ESSCIRC 2022- IEEE 48th European Solid State Circuits Conference (ESSCIRC). 261–264. https://doi.org/10.1109/ESSCIRC55480.2022.9911506

[24]

Vladimir N. Vapnik. 1995. The nature of statistical learning theory. Springer-Verlag New York, Inc.

Digital Library

[25]

J. Wang, M Alea, J van Assche, and G Gielen. 2023. End-to-End Optimization of High-Density e-Skin Design: From Spiking Taxel Readout to Texture Classification. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), in press.

[26]

Colin Weltin-Wu and Yannis Tsividis. 2013. An Event-driven Clockless Level-Crossing ADC With Signal-Dependent Adaptive Resolution. IEEE Journal of Solid-State Circuits 48, 9 (2013), 2180–2190. https://doi.org/10.1109/JSSC.2013.2262738

[27]

P.J. Werbos. 1990. Backpropagation through time: what it does and how to do it. Proc. IEEE 78, 10 (1990), 1550–1560. https://doi.org/10.1109/5.58337

[28]

Minhao Yang, Chen-Han Chien, Tobias Delbruck, and Shih-Chii Liu. 2016. A 0.5V 55μW 64×2-channel binaural silicon cochlea for event-driven stereo-audio sensing. In 2016 IEEE International Solid-State Circuits Conference (ISSCC). 388–389. https://doi.org/10.1109/ISSCC.2016.7418070

Cited By

Chu HYan YZhou YZhu ZHuan YZheng LZou Z(2023)An Energy-Efficient and Robust SNN Classifier for LC-ADC Sampled ECG Signals2023 8th International Conference on Integrated Circuits and Microsystems (ICICM)10.1109/ICICM59499.2023.10365892(502-506)Online publication date: 20-Oct-2023
https://doi.org/10.1109/ICICM59499.2023.10365892

Recommendations

Design space exploration and parameter tuning for neuromorphic applications
CODES+ISSS '13: Proceedings of the Ninth IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis

Large-scale spiking neural networks (SNNs) have been used to successfully model complex neural circuits that explore various neural phenomena such as learning and memory, vision systems, auditory systems, neural oscillations, and many other important ...
Algorithm and Application Impacts of Programmable Plasticity in Spiking Neuromorphic Hardware
ICONS '23: Proceedings of the 2023 International Conference on Neuromorphic Systems

Synaptic plasticity has long been thought to be key to learning in both biological neural networks and artificial neural networks. There are a wide array of plasticity rules in biological neural systems, but neuromorphic computing has mainly focused ...
Fine-tuning with local learning rules helps to compress and accelerate spiking neural networks without accuracy loss
Abstract
Spiking neural networks (SNNs) are believed to be highly energy- and computationally efficient machine learning algorithms, especially when implemented on neuromorphic hardware. Some recent studies have revealed that lateral (intralayer) ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

NICE '23: Proceedings of the 2023 Annual Neuro-Inspired Computational Elements Conference

April 2023

124 pages

ISBN:9781450399470

DOI:10.1145/3584954

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

NICE 2023

NICE 2023: Neuro-Inspired Computational Elements Conference

April 11 - 14, 2023

TX, San Antonio, USA

Acceptance Rates

Overall Acceptance Rate 25 of 40 submissions, 63%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
186
Total Downloads

Downloads (Last 12 months)69
Downloads (Last 6 weeks)4

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chu HYan YZhou YZhu ZHuan YZheng LZou Z(2023)An Energy-Efficient and Robust SNN Classifier for LC-ADC Sampled ECG Signals2023 8th International Conference on Integrated Circuits and Microsystems (ICICM)10.1109/ICICM59499.2023.10365892(502-506)Online publication date: 20-Oct-2023
https://doi.org/10.1109/ICICM59499.2023.10365892

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents