Measuring the Predictability of Recommender Systems using Structural Complexity Metrics
This work addresses the limited understanding of predictability in recommender systems, which is crucial for improving content filtering and curation in online platforms, though it is incremental as it builds on existing methods like SVD and matrix factorization.
The study tackled the problem of measuring the predictability of recommender systems by introducing data-driven metrics based on the structural complexity of user-item rating matrices, and found a high correlation between these metrics and the accuracy of top-performing prediction algorithms on real datasets.
Recommender systems (RS) are central to the filtering and curation of online content. These algorithms predict user ratings for unseen items based on past preferences. Despite their importance, the innate predictability of RS has received limited attention. This study introduces data-driven metrics to measure the predictability of RS based on the structural complexity of the user-item rating matrix. A low predictability score indicates complex and unpredictable user-item interactions, while a high predictability score reveals less complex patterns with predictive potential. We propose two strategies that use singular value decomposition (SVD) and matrix factorization (MF) to measure structural complexity. By perturbing the data and evaluating the prediction of the perturbed version, we explore the structural consistency indicated by the SVD singular vectors. The assumption is that a random perturbation of highly structured data does not change its structure. Empirical results show a high correlation between our metrics and the accuracy of the best-performing prediction algorithms on real data sets.