Motion selectivity of neurons in self-driving networks

Abstract

We investigated if optical flow filters were implicitly learned by a neural network trained to drive a vehicle. The network was not trained to predict optical flow across the frames, but, through a series of controlled experiments, we claim that optical flow filters are present in the network. However, this appears to be only the case for sideways flows more relevant for steering predictions. For motor throttle predictions, the network looks at the variance of the pixels over time rather than computing optical flow. In addition, the filters that are likely used for motor throttle predictions dominate primarily in the middle of the network.