A CMOS vision system on-chip with multicore sensory processing architecture for image analysis above 1,000F/s

Abstract

This paper describes a Vision-System-on-Chip (VSoC) capable of doing: image acquisition, image processing through on-chip embedded structures, and generation of pertinent reaction commands at thousand's frame-per-second rate. The chip employs a distributed processing architecture with a pre-processing stage consisting of an array of programmable sensory-processing cells, and a post-processing stage consisting of a digital microprocessor. The pre-processing stage operates as a retina-like sensor front-end. It performs parallel processing of the images captured by the sensors which are embedded together with the processors. This early processing serves to extract image features relevant to the intended tasks. The front-end incorporates also smart read-out structures which are conceived to transmit only these relevant features, thus precluding full gray-scale frames to be coded and transmitted. The chip is capable to close action-reaction loops based on the analysis of visual flow at rates above 1,000F/s with power budget below 1W peak. Also, the incorporation of processors close to the sensors enables signal-dependent, local adaptation of the sensor gains and hence highdynamic range signal acquisition.