Backpropagation analysis of the limited precision on high-order function neural networks

Abstract

Quantization analysis of the limited precision is widely used in the hardware realization of neural networks. Due to the most neural computations are required in the training phase, the effects of quantization are more significant in this phase. We pay attention and analyze backpropagation training and recall of the limited precision on the HOFNN, point out the potential problems and the performance sensitivity with lower-bit quantization. We compare the training performances with and without weight clipping, derive the effects of the quantization error on backpropagation for on-chip and off-chip training. Our experimental simulation results verify the presented theoretical analysis. © Springer-Verlag 2004.