Evaluation of features to predict the usefulness of online reviews

Abstract

Checking online reviews before purchasing goods or using services has become increasingly common. However, it is difficult to select useful reviews, and concerns about fake reviews are growing. Many online review systems use recency and user-generated usefulness votes in order to prioritize reviews for users, but there is much room for improvement. In this work, we focus on evaluating the effectiveness of a large number of features for predicting the usefulness of online reviews, including features that have not been commonly evaluated in prior work (e.g., social network measures). Features were grouped into hierarchical categories that might represent factors impacting perceived usefulness of Yelp users. Using all features, a binary classifier achieved a high level of accuracy (0.889). Additionally, a feature ablation study found that several feature groups yielded statistically significant improvements. Interestingly, many of the features that improved performance are not the types of measures that are displayed to users in commercial online review services such as Yelp and are the measures that are rarely used to prioritize reviews for users. Our study results suggest different types of information that online review services might want to use in ranking and displaying reviews for users.