Research on spiking neural networks for brain-inspired computing

Abstract

Human brain-like computing is focused on for next-generation artificial intelligence (AI) recently. Spiking neural network (SNN) has its potentials to simulate human brain-like complicated information in relevant to its learning, memory, reasoning, judgment, and decision-making. The higher computational and lower power-consumed SNN can be used to simulate the neurons-biological information transfer mode effectively. It is summarized as mentioned below: first, the brain-biological and event-driven SNN is regarded as a new generation of artificial neural network, which consists of input layer, coding mode, spiking neurons, synaptic weights, learning rules and output layer. SNN has its complication-biological and brain-like mechanism in terms of its spatio-temperal information. Second, its structural optimization is related to 5 aspects and highlighted in terms of 1) coding technique, 2) spiking neuron enhancement, 3) topology, 4) algorithm-trained, and 5) algorithm-coordinated. For the encoding method: 1) grouping, 2) bursting, 3) frequency, and 4) temporal; For the spike neurons-related models: 1) the Hodgkin-Huxley (H-H), 2) the integrated and fire (IF), 3) leaky integrate and fire (LIF), 4) spike response model (SRM), and 5) the Izhikevich between conventional models-relevant. In addition, the improved models for spike neurons are summarized on 8 aspects: for network topology: SNN has 3 conventional structures: 1) feed-forward, 2) recurrent, and 3) hybird on the basis of 5 SNN-based popular topology models. For algorithm-trained, it is summarized from 4 aspects for these methods: 1) back propagation (BP), 2) spike-timing dependent plasticity (STDP), 3) artificial neural network-to-spiking neural network (ANN-to-SNN), and 4) learning algorithms-relevant; Third, the pros and cons of SNN are segmented from the supervised and unsupervised learning both. Supervised learning: since spiking neurons are nonlinear and inconsistent, the error function between the actual output spike sequence and the desired output spike sequence of the neuron is challenged to meet the requirement of differentiability-consistent, resulting in differentiable transfer function and suppress itsback propagation. Unsupervised learning: the Hebb’s rule and STDP algorithm are taken as the key elment in the field of brain-like computing, and it evolves into a variety of learning algorithms. This method is more consistent with the information transmission mode of neurons-biological. But, it is infeasible to build up a large-scale deep network model. Finally, SNNs are applied to brain-like computing and bionic tasks. A systematic SNN-based overview is clarified in the context of its 1) mechanisms, 2) encoding techniques, 3) network structure, and 4) algorithms-trained. Furthermore, to optimize the effectiveness of human brain-like computing model, we predict that the third generation of artificial intelligence is feasible to be realized through information transmission and multi-brain functional simulation and super-large-scale neural network topological and connection-strengthened information memorizing.