A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks

Abstract

To investigate neural network parameters, it is easier to study the distribution of parameters than to study the parameters in each neuron. The ridgelet transform is a pseudo-inverse operator that maps a given function f to the parameter distribution γ so that a network NN[γ] reproduces f, i.e. NN[γ]=f. For depth-2 fully-connected networks on a Euclidean space, the ridgelet transform has been discovered up to the closed-form expression, thus we could describe how the parameters are distributed. However, for a variety of modern neural network architectures, the closed-form expression has not been known. In this paper, we explain a systematic method using Fourier expressions to derive ridgelet transforms for a variety of modern networks such as networks on finite fields Fp, group convolutional networks on abstract Hilbert space H, fully-connected networks on noncompact symmetric spaces G/K, and pooling layers, or the d-plane ridgelet transform.