A Method of discriminative features extraction for restricted boltzmann machines

Abstract

The Restricted Boltzmann Machine (RBM) is a kind of stochastic neural network. It can be used as basic building blocks to form deep architectures. Since Hinton solved the problem of computational inefficiency by using a so called greedy layer-wise unsupervised pretraining algorithm, much more attention is focused on deep learning and achieved significant success in areas of speech recognition, object recognition, natural language processing, etc. In addition to initializing deep networks, RBMs can also be used to learn features from the raw data. In this paper, we proposed a method to learn much better discriminative features for RBMs based on using a novel objective function. We test our idea on MNIST handwritten digit dataset. In our experiments, the features learnt by RBM were further fed to a multinomial logistic regression and results show that our objective function could result in much higher accuracy ratio of classification.