Are Ratings Always Reliable? Discover Users’ True Feelings with Textual Reviews

Abstract

In e-commerce systems, users’ ratings play an important role in many scenarios such as reputation and trust mechanisms and recommender systems. A general assumption in these techniques is that users’ ratings represent their true feelings. Although it has long been adopted in previous work, this assumption is not necessarily true. In this paper, we first present an in-depth study of the inconsistency between users’ ratings and their reviews. Then we propose an approach to mine users’ “true ratings” which better represent their real feelings, from textual reviews based on Gated Recurrent Unit (GRU) and hierarchical attention techniques. One major contribution is that we are about the first, to the best of our knowledge, to investigate this new problem of discovering users’ true ratings, and to provide direct solutions to revise ratings that are insincere and inconsistent. Comparative experiments on a real e-commerce dataset have been conducted, which show that the “true ratings” learned by the proposed model is significantly better than the original ones in terms of consistency with the reviews in three sets of crowdsourcing-based evaluations. Furthermore, leveraging different state-of-art recommendation approaches based on the learned “true ratings”, more effective results have been achieved at all times in rating prediction task.