Neural Network Algorithm with Dropout Using Elite Selection

Abstract

A neural network algorithm is a meta-heuristic algorithm inspired by an artificial neural network, which has a strong global search ability and can be used to solve global optimization problems. However, a neural network algorithm sometimes shows the disadvantage of slow convergence speed when solving some complex problems. In order to improve the convergence speed, this paper proposes the neural network algorithm with dropout using elite selection. In the neural network algorithm with dropout using elite selection, the neural network algorithm is viewed from the perspective of an evolutionary algorithm. In the crossover phase, the dropout strategy in the neural network is introduced: a certain proportion of the individuals who do not perform well are dropped and they do not participate in the crossover process to ensure the outstanding performance of the population. Additionally, in the selection stage, a certain proportion of the individuals of the previous generation with the best performance are retained and directly enter the next generation. In order to verify the effectiveness of the improved strategy, the neural network algorithm with dropout using elite selection is used on 18 well-known benchmark functions. The experimental results show that the introduced dropout strategy improves the optimization performance of the neural network algorithm. Moreover, the neural network algorithm with dropout using elite selection is compared with other meta-heuristic algorithms to illustrate it is a powerful algorithm in solving optimization problems.