Spiking Cooperative Network Implemented on FPGA for Real-Time Event-Based Stereo System

Abstract

A hardware-efficient implementation of a spiking cooperative network (SCN) for a real-time event-based stereo correspondence system is presented. While fully utilizing the advantage of event data, the proposed SCN design significantly reduces the amount of hardware resources by utilizing distinct properties of the SCN, such as the repeatability of synaptic connections and operations, through physical constraints. A stereo system consisting of a field-programmable gate array (FPGA) and a pair of dynamic vision sensors (DVSs) is implemented to demonstrate the SCN design. Stereo livestreamed event data are generated from the DVSs, and the SCN is implemented on an FPGA chip to process the event data. The SCN system has four cores, each comprising an array of 32 Coincidence-Disparity units that calculate the 32-level disparity in a semi-parallel manner. The system performance was evaluated experimentally to estimate the depth of objects moving at different speeds. A rotating drum with a diameter of 8 cm was used in the test. The median relative error of the estimated depth at a rotation speed of 16.7 Hz ranged from 7.3% to 10.6%.