Neural Networks with Quadratic VC Dimension

Abstract

This paper shows that neural networkswhich use continuous activation functions have VC dimension at least as large as the square of the number of weights w. This results settles a long-standing open question, namely whether the well-known O(w log w) bound, known for hard-threshold nets, also held for more general sigmoidal nets. Implications for the number of samples needed for valid generalization are discussed. © 1997 Academic Press.