Fully Complex-Valued Wirtinger Conjugate Neural Networks with Generalized Armijo Search

Abstract

Conjugate gradient (CG) method has been verified to be one effective strategy for training neural networks in terms of its low memory requirements and fast convergence. In this paper, an efficient CG method is proposed to train fully complex neural networks based on Wirtinger calculus. We adopt two ways to enhance the training performance. One is to construct a sufficient descent direction during training by designing a fine tuning conjugate coefficient. Another technique is to pursue the optimal learning rate instead of a fixed constance in each iteration which is determined by employing a generalized Armijo search. To verify the effectiveness and the convergent behavior of the proposed algorithm, the illustrated simulation has been performed on the complex benchmark noncircular signal.