Foveation with optimized receptive fields

Abstract

The sensors of today's artificial vision systems often have millions of pixels. It is a challenge to process this information efficiently and fast. Humans effortlessly handle information from 10**7 photoreceptors, and manage to interact quickly with the environment. To this end, ganglion cells in the retina encode the photoreceptors' responses efficiently by exploiting redundancies in their responses, before sending the information to the visual cortex. Furthermore,primates' ganglion cells develop space-variant properties: their density becomes much higher in the fovea than in the periphery, and the shape and size of the receptive fields vary with the radial distance [1], i.e. primate vision is foveated. Some artificial systems have tried to mimic such foveation to preprocess the visual input [2]. However, these works are based on the photoreceptors' properties instead of those of the ganglion cells, which leads to serious aliasing problems [3]. We propose that artificial systems should implement a model of ganglion cells processing. Our foveation method is formalized as the product between a matrix representing the receptive fields of the ganglion cells and the input image. We combine the information that the distribution of the ganglion cells follows approximately a log-polar law [4] and that the receptive fields have a Difference-of-Gaussian shape [5]. Therefore, each row of the foveation matrix represents a receptive field that depends only on 4 parameters (these are the heights and variances of the two Gaussians: their centres are fixed according to the log-polar density function). We optimize these parameters to reduce the reconstruction error of a generative model using a gradient descent rule (for details see supplementary PDF). We verify that our method converges fast to space variant receptive fields with smaller heigths and size in the fovea than periphery (see supplementary figure 1). We compare the size and shape of the resulting receptive fields with the measures in humans, and discuss about reconstruction optimality in the human early visual system. These results lend themselves to extrapolation to larger image sizes, thereby allowing the implementation of large-scale foveated vision with optimized parameters