A gesture recognition method based on spiking neural networks for cognition development

Abstract

This paper proposes a gesture recognition method based on spiking neural network (SNN). The method can be used to develop the cognition behavior by associating the recognition results with semantic information from the observed target. Firstly, a single shot multi-box detector (SSD) is used to recognize the target object and locate it. Then two SNNs based on Izhikevich model are used to record trajectories of plane motion and depth motion. After projecting and translating the data extracted from the SNN, self-organizing mapping (SOM) and support vector machine (SVM) are applied to realize the gesture recognition. Finally, the associative memory model is used to associate gestures with semantics to achieve cognition. The experiment results show that SNN can well memorize the spatial-temporal information of various gestures. Furthermore, based on the spiking trains from the Izhikevich model, we can realize good results from the clustering and classification.