Ranking with social cues: Integrating online review scores and popularity information

Abstract

Online marketplaces, search engines, and databases employ aggregated social information to rank their content for users. Two ranking heuristics commonly implemented to order the available options are the average review score and item popularity - that is, the number of users who have experienced an item. These rules, although easy to implement, only partly reflect actual user preferences, as people may assign values to both average scores and popularity and trade off between the two. How do people integrate these two pieces of social information when making choices? We present two experiments in which we asked participants to choose 200 times among options drawn directly from two widely used online venues: Amazon and IMDb. The only information presented to participants was the average score and the number of reviews, which served as a proxy for popularity. We found that most people are willing to settle for items with somewhat lower average scores if they are more popular. Yet, our study uncovered substantial diversity of preferences among participants, which indicates a sizable potential for personalizing ranking schemes that rely on social information.