Architecture and Statistical Model of a Pulse-Mode Digital Multilayer Neural Network

Abstract

A new architecture and a statistical model for a pulse-mode digital multilayer neural network (DMNN) are presented. Algebraic neural operations are replaced by stochastic processes using pseudo-random pulse sequences. Synaptic weights and neuron states are represented as probabilities and estimated as average rates of pulse occurrences in corresponding pulse sequences. A statistical model of error (or noise) is developed to estimate relative accuracy associated with stochastic computing in terms of mean and variance. The stochastic computing technique is implemented with simple logic gates as basic computing elements leading to a high neuron-density on a chip. Furthermore, the use of simple logic gates for neural operations, the pulse-mode signal representation, and the modular design techniques lead to a massively parallel yet compact and flexible network architecture, well suited for VLSI (very large scale integration) implementation. Any size of a feedforward network can be configured where processing speed is independent of the network size. Multilayer feedforward networks are modeled and applied to pattern classification problems such as encoding and character recognition. The architecture and all digital subcomponents in the proposed neural network are modeled and simulated in VHDL (VHSIC hardware description language). Computational accuracy is analyzed and the network performance is evaluated in terms of correct classification rates. Experiments show the network performance is competitive to a deterministic calculation while retaining the advantages of high density and high speed. © 1995 IEEE