Analysis and Synthesis of Feedforward Neural Networks Using Discrete Affine Wavelet Transformations

Abstract

In this paper we develop a representation of a class of feedforward neural networks in terms of discrete affine wavelet transforms. It is shown that by appropriate grouping of terms, feedforward neural networks with sigmoidal activation functions can be viewed as architectures which implement affine wavelet decompositions of mappings. This result follows simply from the observation that standard feedforward network architectures possess an inherent translation-dilation structure and every node implements the same activation function. It is shown that the wavelet transform formalism provides a mathematical framework within which it is possible to perform both analysis and synthesis of feedforward networks. For the purpose of analysis, the wavelet formulation characterizes a class (I2) of mappings which can be implemented by feedforward networks as well as reveals an exact implementation of a given mapping in this class. Spatio-spectral localization properties of wavelets can be exploited in synthesizing a feedforward network to perform a given approximation task. Synthesis procedures based on spatio-spectral localization result in reducing the training problem to one of convex optimization. We outline two such synthesis schemes. © 1993 IEEE