Determining Optimal Number of Hidden Units for Multi-Layered Neural Networks

Abstract

Multi-layered neural networks have been widely applied to control and recognition Systems. However, since there is no deterministic method to obtain an optimal number of hidden units, trial and error Simulation is required to develop these Systems, which is very time consuming. In this paper, a Statistical algorithm based on linear regression analysis is proposed to determine the optimal number of hidden units. The optimal number defined here is the smallest one which is sufficient for the maximum generalization capability of the network. By using a trained network with enough number of hidden units, the developed method calculates the amount of linear correlation between their Outputs and estimates the redundant number of hidden units corresponding to it. Finally by subtracting the redundant number from the initial one, the optimal number of hidden units can be obtained. The developed method is applied to two typical neural network Systems, i. e. a Classification System (number recognition) and a non-linear approximation System (coagulant injection Operation in a water purification plant), and demonstrated to be effective to estimate the optimal number of hidden units. © 1992, The Institute of Electrical Engineers of Japan. All rights reserved.