Abstract
In the field of brain-machine interface (BMI), deep learning algorithms have been steadily advancing as the go-to instrument for the key task of neural decoding. However, to function in real-time on portable devices, these algorithms must adhere to stringent limitations on computational power and memory. In this work, we exploit spiking neural networks (SNNs) within a real-time neural decoding system deployed on a low-end Artix-7 FPGA. The system is capable of decoding the spike activity in intracortical neural signals, recorded by a 96-channels microelectrode array, to continuously and concurrently track five target variables in a reach-to-grasp experiment. We have assessed our approach on a widely used reference dataset, achieving a decoding accuracy comparable with alternatives in literature, which exploit more complex deep learning models on the same dataset to decode a single target variables. Our system uses around 20 times less parameters than other non-SNN approaches and consumes 56.4 mW.
The authors acknowledge funding from Sardegna Ricerche, Bando “PROOF of CONCEPT - Valorizzazione dei risultati della ricerca in biomedicina” - PO FESR 2014–2020 - Deep-ECGEE project.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Petrini, F.M., et al.: Six-month assessment of a hand prosthesis with intraneural tactile feedback. Ann. Neurol. 85(1), 137–154 (2019)
Moses, D.A., et al.: Neuroprosthesis for decoding speech in a paralyzed person with anarthria. N. Engl. J. Med. 385(3), 217–227 (2021)
Nurmikko, A.: Challenges for large-scale cortical interfaces. Neuron 108(2), 259–269 (2020)
Busia, P., et al.: EEGformer: transformer-based epilepsy detection on raw EEG traces for low-channel-count wearable continuous monitoring devices. In: 2022 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 640–644 (2022). https://doi.org/10.1109/BioCAS54905.2022.9948637
Wang, X., Lin, X., Dang, X.: Supervised learning in spiking neural networks: A review of algorithms and evaluations. Neural Netw. 125, 258–280 (2020)
Shrestha, S.B., Orchard, G.: Slayer: Spike layer error reassignment in time. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Lee, C., Sarwar, S.S., Panda, P., Srinivasan, G., Roy, K.: Enabling spike-based backpropagation for training deep neural network architectures. Front. Neurosci., 119 (2020)
Yan, Z., Zhou, J., Wong, W.-F.: Energy efficient ECG classification with spiking neural network. Biomed. Signal Process. Control 63, 102170 (2021)
Xiping, J., Fang, B., Yan, R., Xiaoliang, X., Tang, H.: An FPGA implementation of deep spiking neural networks for low-power and fast classification. Neural Comput. 32(1), 182–204 (2020)
Sun, B., Feng, H., Chen, K., Zhu, X.: A deep learning framework of quantized compressed sensing for wireless neural recording. IEEE Access 4, 5169–5178 (2016). https://doi.org/10.1109/ACCESS.2016.2604397
Leone, G., Raffo, L., Meloni, P.: ZyON: enabling spike sorting on APSoC-based signal processors for high-density microelectrode arrays. IEEE Access 8, 218145–218160 (2020). https://doi.org/10.1109/ACCESS.2020.3042034
Milekovic, T., Truccolo, W., Grün, S., Riehle, A., Brochier, T.: Local field potentials in primate motor cortex encode grasp kinetic parameters. Neuroimage 114, 338–355 (2015)
Tagliabue, M., et al.: Estimation of two-digit grip type and grip force level by frequency decoding of motor cortex activity for a BMI application. In: 2015 International Conference on Advanced Robotics (ICAR), pp. 308–315. IEEE (2015)
Khorasani, A., Heydari Beni, N., Shalchyan, V., Daliri, M.R.: Continuous force decoding from local field potentials of the primary motor cortex in freely moving rats. Sci. Rep. 6(1), 1–10 (2016)
Ahmadi, N., Constandinou, T.G., Bouganis, C.S.: Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning. J. Neural Eng. 18(2), 026011 (2021)
Yang, S.-H., Huang, J.-W., Huang, C.-J., Chiu, P.-H., Lai, H.-Y., Chen, Y.-Y.: Selection of essential neural activity timesteps for intracortical brain-computer interface based on recurrent neural network. Sensors 21(19), 6372 (2021)
Leone, G., Raffo, L., Meloni, P.: On-FPGA spiking neural networks for end-to-end neural decoding. IEEE Access 11, 41387–41399 (2023). https://doi.org/10.1109/ACCESS.2023.3269598
Ma, X., Zheng, W., Peng, Z., Yang, J.: FPGA-based rapid electroencephalography signal classification system. In: 2019 IEEE 11th International Conference on Advanced Infocomm Technology (ICAIT), pp. 223–227. IEEE (2019)
Agrawal, M., Vidyashankar, S., Huang, K.: On-chip implementation of ECoG signal data decoding in brain-computer interface. In: 2016 IEEE 21st International Mixed-Signal Testing Workshop (IMSTW), pp. 1–6. IEEE (2016)
McCrimmon, C.M., et al.: Performance assessment of a custom, portable, and low-cost brain-computer interface platform. IEEE Trans. Biomed. Eng. 64(10), 2313–2320 (2017)
Du, Y., Jin, J., Wang, Q., Fan, J.: EMG-based continuous motion decoding of upper limb with spiking neural network. In: 2022 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), pp. 1–5. IEEE (2022)
Liao, J., et al.: An energy-efficient spiking neural network for finger velocity decoding for implantable brain-machine interface. In: 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 134–137. IEEE (2022)
Leone, G., Raffo, L., Meloni, P.: A bandwidth-efficient emulator of biologically-relevant spiking neural networks on FPGA. IEEE Access 10, 76780–76793 (2022). https://doi.org/10.1109/ACCESS.2022.3192826
Furber, S.B., Galluppi, F., Temple, S., Plana, L.A.: The spinnaker project. Proc. IEEE 102(5), 652–665 (2014)
Brochier, T., et al.: Massively parallel recordings in macaque motor cortex during an instructed delayed reach-to-grasp task. Sci. Data 5(1), 1–23 (2018)
Davies, M., et al.: Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1), 82–99 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Leone, G., Martis, L., Raffo, L., Meloni, P. (2023). On-FPGA Spiking Neural Networks for Multi-variable End-to-End Neural Decoding. In: Palumbo, F., Keramidas, G., Voros, N., Diniz, P.C. (eds) Applied Reconfigurable Computing. Architectures, Tools, and Applications. ARC 2023. Lecture Notes in Computer Science, vol 14251. Springer, Cham. https://doi.org/10.1007/978-3-031-42921-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-42921-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42920-0
Online ISBN: 978-3-031-42921-7
eBook Packages: Computer ScienceComputer Science (R0)