Evaluating Music Recommendations with Binary Feedback for Multiple Stakeholders
This work addresses the challenge of balancing multiple stakeholders in music recommendations, but it is incremental as it applies an existing method to a new dataset.
The study tackled the problem of evaluating music recommendation systems with noisy feedback data by using a dataset of 500k ratings to assess performance for consumers, well-known artists, and lesser-known artists, finding that a matrix factorization algorithm trained on both likes and dislikes performed significantly better than one trained only on likes for all stakeholders.
High quality user feedback data is essential to training and evaluating a successful music recommendation system, particularly one that has to balance the needs of multiple stakeholders. Most existing music datasets suffer from noisy feedback and self-selection biases inherent in the data collected by music platforms. Using the Piki Music dataset of 500k ratings collected over a two-year time period, we evaluate the performance of classic recommendation algorithms on three important stakeholders: consumers, well-known artists and lesser-known artists. We show that a matrix factorization algorithm trained on both likes and dislikes performs significantly better compared to one trained only on likes for all three stakeholders.