Ordinal Regression for Difficulty Estimation of StepMania Levels
This work addresses the challenge for players and level authors in determining difficulty levels of community contributions in StepMania, representing an incremental improvement with domain-specific impact.
The paper tackled the problem of predicting difficulty levels for community-designed StepMania rhythm game levels by formalizing it as an ordinal regression task, resulting in neural network models that significantly outperformed the state of the art and demonstrated superiority over human labeling in a user experiment.
StepMania is a popular open-source clone of a rhythm-based video game. As is common in popular games, there is a large number of community-designed levels. It is often difficult for players and level authors to determine the difficulty level of such community contributions. In this work, we formalize and analyze the difficulty prediction task on StepMania levels as an ordinal regression (OR) task. We standardize a more extensive and diverse selection of this data resulting in five data sets, two of which are extensions of previous work. We evaluate many competitive OR and non-OR models, demonstrating that neural network-based models significantly outperform the state of the art and that StepMania-level data makes for an excellent test bed for deep OR models. We conclude with a user experiment showing our trained models' superiority over human labeling.