BiRating -- Iterative averaging on a bipartite graph of Beat Saber scores, player skills, and map difficulties
This work addresses difficulty estimation for Beat Saber maps, which is valuable to the competitive gaming community, but it is incremental as it builds on existing bipartite graph methods for skill/difficulty estimation.
The researchers tackled the problem of estimating Beat Saber map difficulties by developing an iterative averaging algorithm on a bipartite graph of player scores, which simultaneously estimates player skills and map difficulties. The approach showed significant alignment with player perceptions and improvement over existing methods on problematic maps, though it produced problematic estimations for certain map families where score assumptions were inadequate.
Difficulty estimation of Beat Saber maps is an interesting data analysis problem and valuable to the Beat Saber competitive scene. We present a simple algorithm that iteratively averages player skill and map difficulty estimations in a bipartite graph of players and maps, connected by scores, using scores only as input. This approach simultaneously estimates player skills and map difficulties, exploiting each of them to improve the estimation of the other, exploitng the relation of multiple scores by different players on the same map, or on different maps by the same player. While we have been unable to prove or characterize theoretical convergence, the implementation exhibits convergent behaviour to low estimation error in all instances, producing accurate results. An informal qualitative evaluation involving experienced Beat Saber community members was carried out, comparing the difficulty estimations output by our algorithm with their personal perspectives on the difficulties of different maps. There was a significant alignment with player perceived perceptions of difficulty and with other existing methods for estimating difficulty. Our approach showed significant improvement over existing methods in certain known problematic maps that are not typically accurately estimated, but also produces problematic estimations for certain families of maps where the assumptions on the meaning of scores were inadequate (e.g. not enough scores, or scores over optimized by players). The algorithm has important limitations, related to data quality and meaningfulness, assumptions on the domain problem, and theoretical convergence of the algorithm. Future work would significantly benefit from a better understanding of adequate ways to quantify map difficulty in Beat Saber, including multidimensionality of skill and difficulty, and the systematic biases present in score data.