Stochastic rounding for memory-efficient digital simulation of synaptic plasticity using 8-bit floating-point

Abstract

Simulating brain-scale networks digitally is often hindered by extensive memory access. In this context, using low-precision data types and more efficient models to represent state variables is a viable alternative to improve the scalability of the networks under consideration. However, understanding whether these approaches hurt the expected dynamics of the system is critical yet still poorly understood. This study aims to integrate 8-bit floating-point implementations with neural models optimised for efficient memory access to simulate spiking neural networks endowed with spike-timing-dependent plasticity. By doing this, we not only address scalability issues but also consider the biological plausibility of the network activity. We show that stochastic rounding (SR) is necessary to overcome the floating-point errors associated with weight updates and present our custom SR scheme, which was applied to all arithmetic operations. Under these conditions, we study the limitations and behaviour of plastic weight dynamics and large-scale balanced networks. Our results suggest that the models developed can be used to reproduce many prominent features previously described in studies of cortical stability and information processing. Such an approach offers a promising perspective on optimising spiking neural networks for real-time simulations and resource-constrained digital hardware. Such technology could be used to understand neurological disorders by providing insights into abnormal neural activity patterns. It could also enhance brain-computer interfaces, and aid in cognitive neuroscience research, contributing to novel therapeutic strategies.