Statistical Ideas for Selecting Network Architectures

Abstract

Choosing the architecture of a neural network is one of the most important prob- lems in making neural networks practically useful, but accounts of applications usually sweep these details under the carpet. How many hidden units are needed? Should weight decay be used, and if so how much? What type of output units should be chosen? And so on.We address these issues within the framework of statistical theory for model choice, which provides a number of workable approximate answers.