A comparison study on ensemble strategies and feature sets for sentiment analysis

Abstract

This paper is devoted to the comparison of different common base and ensemble classifiers for sentiment classification of reviews. It is also aimed to generate different feature sets and to observe their contribution to the classification accuracy. In detail, these feature sets are formed in an hierarchical manner, which is accomplished by first forming part-of-speech (POS) based word groups and then utilizing feature frequencies, SentiWordNet scores and their combination to obtain feature sets. In addition, several common base classifiers, namely Multinominal Naive Bayes (MNB), Support Vector Machine (SVM), Voted Perceptron (VP), K-Nearest Neighbor (k-NN), as well as common ensemble strategies, Random Forests (RFs), Stacking and Random Subspace (RSS) are each tested on the generated feature sets. Also, the Behavior-Knowledge Space (BKS) method has been derived to be applied on the set of outcomes for different algorithm and feature set combinations. Furthermore, a probability based meta-classifier technique has been tested on this set of outcomes. Finally, Information Gain (IG) feature selection technique has been applied to reduce the feature spaces. The experiments are conducted on a widely used movie review dataset and an equally common multi-domain review dataset. The results indicate that the probabilistic ensemble method generally gives comparatively better results than the other algorithms tested on the chosen datasets and that IG method can be utilized to save computational time while maintaining allowable accuracy.