Quantitative analysis of Matthew effect and sparsity problem of recommender systems
This addresses data structure challenges for recommender system developers, but it is incremental as it builds on existing collaborative filtering frameworks.
The paper tackles the Matthew effect and sparsity problems in recommender systems by conducting a quantitative analysis within collaborative filtering, comparing user-based and item-based methods to provide insights for industrial builders.
Recommender systems have received great commercial success. Recommendation has been used widely in areas such as e-commerce, online music FM, online news portal, etc. However, several problems related to input data structure pose serious challenge to recommender system performance. Two of these problems are Matthew effect and sparsity problem. Matthew effect heavily skews recommender system output towards popular items. Data sparsity problem directly affects the coverage of recommendation result. Collaborative filtering is a simple benchmark ubiquitously adopted in the industry as the baseline for recommender system design. Understanding the underlying mechanism of collaborative filtering is crucial for further optimization. In this paper, we do a thorough quantitative analysis on Matthew effect and sparsity problem in the particular context setting of collaborative filtering. We compare the underlying mechanism of user-based and item-based collaborative filtering and give insight to industrial recommender system builders.