Mutation operators for structure evolution of neural networks

Abstract

The architecture of a neural network can be optimized by structure evolution. The structure evolution is based upon a two-stage evolution strategy (multipopulation strategy): On the population level, the structure is optimized, on the individual level, the parameters are adapted. For the variation of the (discrete) architecture, a mutation operator must be defined. To attain successful optimization, a mutation operator must satisfy two main conditions in the space of structures: First the principle of strong causality must be obeyed (smoothness of the fitness landscape in the space of structure), and second, a transition path between the structures must be guaranteed. In this paper different heuristic mutation operators will be defined and examined on their behavior with respect to strong causality and to neighborhood relation in the space of structures.