Ranking with social cues: Integrating online review scores and popularity information
This addresses the problem of improving ranking algorithms for online marketplaces and search engines by personalizing based on user preferences, though it is incremental as it builds on existing social cues.
The study investigated how people integrate average review scores and popularity (number of reviews) when making choices, using data from Amazon and IMDb, and found that most participants prefer more popular items even with slightly lower scores, revealing diverse preferences.
Online marketplaces, search engines, and databases employ aggregated social information to rank their content for users. Two ranking heuristics commonly implemented to order the available options are the average review score and item popularity-that is, the number of users who have experienced an item. These rules, although easy to implement, only partly reflect actual user preferences, as people may assign values to both average scores and popularity and trade off between the two. How do people integrate these two pieces of social information when making choices? We present two experiments in which we asked participants to choose 200 times among options drawn directly from two widely used online venues: Amazon and IMDb. The only information presented to participants was the average score and the number of reviews, which served as a proxy for popularity. We found that most people are willing to settle for items with somewhat lower average scores if they are more popular. Yet, our study uncovered substantial diversity of preferences among participants, which indicates a sizable potential for personalizing ranking schemes that rely on social information.