Optimization of neural networks using deep genetic network algorithm

Abstract

The optimization of performance of a neural network is a time taking and tedious process, this iterative and con-tinuous process has no definite solution that works well for every possible use case. To tackle this problem we propose an architecture of neural networks called "Deep Genetic Network", which can help in automatic selection of hyper parameter values based on fitness measures during training of the network. The algorithm is a confluence of deep neural networks and genetic algo-rithm. The problem of optimizing a neural network can be classi-fied into-Architecture and Hyperparameter optimization. A va-riety of algorithms have been proposed to solve this issue. Our approach uses concepts of mutation and mating (from genetic algorithms) for helping the neural net in finding the optimal set of hyperparameter values during training without requiring any manually setting the values in an iterative trial and error ap-proach. The architecture that we propose here works well in optimization of hyperparameter values in convolutional, recurrent and affine layers. The usage of genetic algorithms for resolving this issue has worked well given adequate training time and com-putational resources.