Approximation and learning of convex superpositions

Abstract

We present a fairly general method for constructing classes of functions of finite scale-sensitive dimension (the scale-sensitive dimension is a generalization of the Vapnik-Chervonenkis dimension to real-valued functions). The construction is as follows: start from a class F of functions of finite VC dimension, take the convex hull coF of F, and then take the closure coF of coF in an appropriate sense. As an example, we study in more detail the case where F is the class of threshold functions. It is shown that coF includes two important classes of functions: neural networks with one hidden layer and bounded output weights; the so-called r class of Barron, which was shown to satisfy a number of interesting approximation and closure properties. We also give an integral representation in the form of a "continuous neural network" which generalizes Barron's. It is shown that the existence of an integral representation is equivalent to both I2 and L°° approx-imability.