Information geometry on modular and hierarchical neural network

Abstract

Although theoretic research and applications have proved that modular neural network (NN) and hierarchical NN have notable effect to solve the complex problem, there is still little work on a mathematical framework about their structure mechanism. This paper gives a theoretic analysis based on information geometry. By studying the dual manifold architecture for modular NN and hierarchical NN and analyzing the probability of knowledge increasable based on information geometry, the paper also proposes a new modular and hierarchical model: multi-HME that has knowledge-increasable and structure-extendible ability. © Springer-Verlag Berlin Heidelberg 2003.