Divergence projections for variable selection in multi-layer perceptron networks

Abstract

In this paper an information geometric-based variable selection method for MLP networks is shown. It is based on divergence projections of the Riemannian manifold defined by a MLP network on submanifolds defined by MLP networks with reduced input dimension. We show how we can take advantage of the layered structure of the MLP to simplify the projection operation, which cannot be accurately done by using only the Fisher information metric. Furthermore, we show that our selection algorithm is more robust and gives better results than other well known selection algorithms like Optimal Brain Surgeon. Some examples are shown to validate the proposed approach. © Springer-Verlag Berlin Heidelberg 2003.