Probing a Deep Neural Network

Abstract

We report a number of experiments on a deep convolutional network in order to gain a better understanding of the transformations that emerge from learning at the various layers. We analyze the backward flow and the reconstructed images, using an adaptive masking approach in which pooling and nonlinearities at the various layers are represented by data-dependent binary masks. We focus on the field of view of specific neurons, also using random parameters, in order to understand the nature of the information that flows through the activation’s“holes" that emerge in the multi-layer structure when an image is presented at the input. We show how the peculiarity of the multi-layer structure is not so much in the learned parameters, but in the patterns of connectivity that are partly imposed and then learned. Furthermore, a deep network appears to focus more on statistics, such as gradient-like transformations, rather than on filters matched to image patterns. Our probes seem to explain why classical image processing algorithms, such as the famous SIFT, have provided robust, although limited, solutions to image recognition tasks.