A novel text classification approach based on deep belief network

Abstract

A novel text classification approach is proposed in this paper based on deep belief network. Deep belief network constructs a deep architecture to obtain the high level abstraction of input data, which can be used to model the semantic correlation among words of documents. After basic features are selected by statistical feature selection measures, a deep belief network with discriminative fine tuning strategy is built on basic features to learn high level deep features. A support vector machine is then trained on the learned deep features. The proposed method outperforms traditional classifier based on support vector machine. As a dimension reduction strategy, the deep belief network also outperforms the traditional latent semantic indexing method. Detailed experiments are also made to show the effect of different fine tuning strategies and network structures on the performance of deep belief network. © 2010 Springer-Verlag.