Selected problems of artificial neural networks development

Abstract

The chapter discusses selected problems of applications of Standard (deterministic) Neural Networks (SNN) but the main attention is focused on Bayesian Neural Networks (BNNs). In Sections 2 and 3 the problems of regression analysis, over-fitting and regularization are discussed basing on two types of network, i.e. Feed-forward Layered Neural Network (FLNN) and Radial Basis Function NN (RBFN). Application of Principal Component Analysis (PCA) is discussed as a method for reduction of input space dimensionality. In Section 4 the application of Kalman filtering to learning of SNNs is presented. Section 5 is devoted to discussion of some basics related to Bayesian inference. Then Maximum Likelihood (ML) and Maximum A Posterior (MAP) methods are presented as a basis for formulation of networks SNN-ML and SNN-MAP. A more general Bayesian framework corresponding to formulation of simple, semi-probabilistic network S-BNN, true probabilistic T-BNN and Gaussian Process GP-BNN is discussed. Section 6 is devoted to the analysis of four study cases, related mostly to the analysis of structural engineering and material mechanics problems.