Hidden-layer size reducing for multilayer neural networks using the orthogonal least-squares method

Abstract

This paper proposes a new approach to hidden-layer size reducing for multilayer neural networks, using the orthogonal least-squares (OLS) method with the aid of Gram-Schmidt orthogonal transformation. A neural network with a large hidden-layer size is first trained via a standard training rule. Then the OLS method is introduced to identify and eliminate redundant neurons such that a simpler neural network is obtained. The OLS method is employed as a forward regression procedure to select a suitable set of neurons from a large set of preliminarily trained hidden neurons, such that the input to the output-layer's neuron is reconstructed with less hidden neurons. Simulation results are included to show the efficiency of the proposed method.