Dynamic Vision Sensor Based Gesture Recognition Using Liquid State Machine

Abstract

Dynamic Vision Sensor (DVS) is a neuromorphic sensor. Compared with traditional cameras, event cameras based on DVS have the advantages of high time resolution, low power consumption, less motion blur, and low data redundancy, making DVS more suitable for computer vision tasks such as gesture recognition. However, the adaptation of traditional algorithms to computer vision tasks with DVS seems to have unsatisfying limitations. This paper implements a lightweight gesture recognition system using Liquid State Machine (LSM). LSM can use the events generated by DVS as input directly. To achieve better performance of LSM, we use a heuristic search algorithm to obtain an improved parameter configuration for LSM. Our system achieves 98.42% accuracy on the DVS128 Gesture Dataset. Our model contains remarkably 90% fewer parameters than the Inception 3D, which achieves a precision of 99.62% on the DVS128 Gesture Dataset. This work encodes and compresses the event stream into event frames by using Convolutional Neural Network (CNN) to extract features from the frame and then sending the extracted feature to reservoirs to implement the gesture recognition.