Studying the Effect of Optimizing Weights in Neural Networks with Meta-Heuristic Techniques

Abstract

Meta-heuristic algorithms provide derivative-free solutions to optimize complex problems. Back-propagation Neural Network (BP) algorithm is one of the most commonly used and a popular technique to optimize the feed forward neural network training. Traditional BP algorithm has some drawbacks, such as getting stuck easily in local minima and slow speed of convergence. This paper proposed a new meta-heuristic search algorithm, called cuckoo search (CS), based on cuckoo bird’s behavior to train back propagation (BP), Elman Recurrent Neural Network (RNN), and Levenberg Marquardt (LM) algorithms to achieving fast convergence rate and to avoid local minima problem. The performances of the proposed hybrid Cuckoo Search algorithms are compared with artificial bee colony using BP algorithm, and other hybrid variant. Specifically on Iris and 7-Bit parity datasets are used. The simulation results show that the hybrid Cuckoo Search show better performances than the other hybrid technique.