One-Hot Multi-Level Leaky Integrate-and-Fire Spiking Neural Networks for Enhanced Accuracy-Latency Tradeoff

Abstract

Spiking neural networks (SNNs) hold significant promise as energy-efficient alternatives to conventional artificial neural networks (ANNs). However, SNNs require computations across multiple timesteps, resulting in increased latency, heightened energy consumption, and additional memory access overhead. Techniques to reduce SNN latency down to a unit timestep have emerged to realize true superior energy efficiency over ANNs. Nonetheless, this latency reduction often comes at the expense of noticeable accuracy degradation. Therefore, achieving an optimal balance in the tradeoff between accuracy and energy consumption by adjusting the latency of multiple timesteps remains a significant challenge. This work leverages an additional dimension to enhance the accuracy-energy tradeoff space using a novel one-hot multi-level leaky integrate-and-fire (M-LIF) neuron model. The proposed one-hot M-LIF model represents the inputs and outputs of hidden layers as a set of one-hot binary-weighted spike lanes to find better tradeoff points while still being able to model conventional SNNs. For image classification on static datasets, we demonstrate one-hot M-LIF SNNs outperform iso-architecture conventional LIF SNNs in terms of accuracy (2% higher than VGG16 SNN on ImageNet) while still being energy-efficient (20× lower energy than VGG16 ANN on ImageNet). For dynamic vision datasets, we demonstrate the ability of M-LIF SNNs to reduce latency by 3× compared to conventional LIF SNNs while limiting accuracy degradation (<1%).