A method of pruning layered feed-forward neural networks

Abstract

The problem of reducing the size of a trained multilayer artificial neural network is addressed, and a method of removing hidden units is developed. The method is based on the idea of eliminating units and adjusting remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of a system of linear equations, and a very efficient conjugate-gradient algorithm is used for solving it, in the least squares sense. The algorithm also provides a sub-optimal criterion for choosing the units to be removed, which is proved lo work well in practice. Preliminary results over a simulated pattern recognition task are reported, which demonstrate the effectiveness of the proposed approach.