Improvement of the backpropagation algorithm using (1+1) evolutionary strategies

Abstract

Currently, the standard in supervised Artificial Neural Networks (ANNs) research is to use the backpropagation (BP) algorithm or one of its improved variants, for training. In this chapter, we present an improvement to the most widely used BP learning algorithm using (1+1) evolutionary Strategy (ES), one of the most widely used artificial evolution paradigms. The goal is to provide a method that can adaptively change the main learning parameters of the BP algorithm in an unconstrained manner. The BP/ES algorithm we propose is simple to implement and can be used in combination with various improved versions of BP. In our experimental tests we can see a substantial improvement in ANN performance, in some cases a reduction of more than 50% in error for time series prediction on a standard benchmark test. Therefore, we believe that our proposal effectively combines the learning abilities of BP with the global search of ES to provide a useful tool that improves the quality of learning for BP-based methods. © 2010 Springer-Verlag Berlin Heidelberg.