A Specific and Selective Neural Response Representation with Decorrelating Auto-Encoder

Abstract

Since the pioneering report with an unsupervised pre-training principle was published, deep architectures, as a simulation of primary cortexes, have been intensively studied and successfully utilized in solving some recognition tasks. Motivated by that, herein, we propose a decorrelating regularity on auto-encoders, named decorrelating auto-encoder (DcA), which can be stacked to deep architectures, called the SDcA model. The learning algorithm is designed based on the principles of redundancy-reduction and the infomax, and a fine-tuning algorithm based on correlation detecting criteria. The property of our model is evaluated by auditory and handwriting recognition tasks with the TIMIT acoustic-phonetic continuous speech corpus and MNIST database. The results show that our model has a general advantage as compared with four existing models, especially in low levels, and when training samples are scarce our model put up stronger learning capacity and generalization.