Mathematical Modeling of Product Rating: Sufficiency, Misbehavior and Aggregation Rules
This addresses the challenge of reliable product evaluation for online platforms and consumers, but it is incremental as it builds on existing rating aggregation methods.
The paper tackles the problem of evaluating product quality from partial user ratings, deriving theoretical bounds on the minimum ratings needed for reliability and showing that a majority rating rule outperforms averaging in robustness, with validation on real-world data from TripAdvisor, Amazon, and eBay.
Many web services like eBay, Tripadvisor, Epinions, etc, provide historical product ratings so that users can evaluate the quality of products. Product ratings are important since they affect how well a product will be adopted by the market. The challenge is that we only have {\em "partial information"} on these ratings: Each user provides ratings to only a "{\em small subset of products}". Under this partial information setting, we explore a number of fundamental questions: What is the "{\em minimum number of ratings}" a product needs so one can make a reliable evaluation of its quality? How users' {\em misbehavior} (such as {\em cheating}) in product rating may affect the evaluation result? To answer these questions, we present a formal mathematical model of product evaluation based on partial information. We derive theoretical bounds on the minimum number of ratings needed to produce a reliable indicator of a product's quality. We also extend our model to accommodate users' misbehavior in product rating. We carry out experiments using both synthetic and real-world data (from TripAdvisor, Amazon and eBay) to validate our model, and also show that using the "majority rating rule" to aggregate product ratings, it produces more reliable and robust product evaluation results than the "average rating rule".