OPTIMIZING RATE-BASED SPIKING NEURAL NETWORKS

  • Chen Li

Student thesis: Phd

Abstract

A notable trend in recent years is the transition of the prevalent deep learning algorithms to edge devices, and a primary concern is unsustainable energy dissipation. Deep SNN, benefiting from their event-based nature and efficient information communication by spikes, can serve as a competitive candidate for achieving a more power-efficient computing paradigm. A practical way to train an SNN is to first train an ANN and then convert it into a rate-coded SNN, a method called ANN-to-SNN conversion. This method enables building functional SNNs at a low cost and validating various optimization strategies in SNNs. Based on ANN-to-SNN conversion, this thesis explores the rationale behind SNNs and the optimization of SNNs from various aspects. First, it clarifies the fundamental question of why to use SNNs. Few advantages of SNNs compared with conventional ANN have been found up to now. The presented results show that SNNs can render better robustness to noisy synaptic weights. This research paves the way for applying memristors, a cutting-edge component with intrinsic noise, to spike-based in-memory computing. Second, it focuses on retaining the biological plausibility of state-of-the-art SNNs. In the presented study, the neuronal dynamics of the standard integrate-and-fire model are analyzed, and the difficulty of weight-bias imbalance when using this model is relieved. Better accuracy is achieved than the state-of-the-art SNNs. Third, the accuracy latency trade-off, one of the essential challenges in rate-coded SNNs, is alleviated in the presented study. It elaborates on the role of noise in fast SNNs and the necessity of information compression in achieving low-latency SNNs. The SNNs optimized by this approach achieved an accuracy of 70.18% in 8 times steps on ImageNet. Finally, SNNs need to be deployed to neuromorphic hardware or neuromorphic chips for real-world applications. An SNN deployment on SpiNNaker is described in this thesis, featuring high accuracy (98.63% on MNIST), structural plasticity, and low firing rates.
Date of Award1 Aug 2023
Original languageEnglish
Awarding Institution
  • The University of Manchester
SupervisorOliver Rhodes (Supervisor) & Steve Furber (Supervisor)

Cite this

'