A new gradient-based search method: Grey-gradient search method

Abstract

Optimization theory and methods play very important role for engineering design and applications. In many domains of engineering applications, it is usually the most important process to find near optimal solution. The gradient-descent method is widely used to solve many engineering optimization problems. But the gradient-descent method has some disadvantages for searching optimal solution. Firstly, its convergent speed is very slowly and is easy to trap into local minimum in the applications of many actual problems. Secondly, the learning rate of gradient-descent method must been determined adequately for different engineering problem. If the learning rate set very small, the convergent speed will be very slowly. If the learning rate is set very large, the searching of solution is very easy to generate trashing or divergence. The main goal of this research is to propose a new method that is based on grey prediction theory to improve the gradient-descent method. We use the idea of grey prediction to speed up effectively the searching speed of gradient-descent method, and improve the drawback that gradient-descent method is very easy trap into local minimum. From the experimental results, we can show the workings of the proposed method that can speed up effectively the searching speed of gradient-descent method, and improve the drawback that gradientdescent method is easy to trapped into local minimum.