A Review of the Theory and Method for New Developed Feedforward Neural Networks

Abstract

Two new theories and methods of feedforward neural network developed in recent years are systematically reviewed. They are Algebraic Algorithm of feedforward neural network and Spline Weight Function Algorithm of feedforward neural network. The network structure, basic theory and algorithm, and basic performance of the two networks are described. The algebraic algorithm of feedforward neural network can get the global optimum, but BP algorithm is difficult to get the global optimum. Algebraic algorithm has high accuracy and fast training speed. In engineering application, the algebraic algorithm gives the calculation formula of accurately determining the number of hidden layer neurons, but the BP algorithm does not give such an accurate formula. However, algebraic algorithm, like traditional BP algorithm, obtains constant weights, which is difficult to reflect the information of training samples. The spline weight function algorithm of feedforward neural network not only retains the advantages of algebraic algorithm, but also overcomes its shortcomings, so that the weight of trained neural network is a function of input sample (called weight function), which is composed of spline function, not a constant of traditional method. The trained weight function can extract useful information features from the trained samples. The topological structure of the spline weighted neural network is very simple, and the number of its neurons is independent of the number of samples, which is equal to the number of input and output nodes. The calculation speed is fast. Moreover, with the increase of the number of samples, the generalization ability of the network is also improving.