FSBench: A Figure Skating Benchmark for Advancing Artistic Sports Understanding
This addresses the problem of limited research tools for artistic sports, providing a benchmark for evaluating AI models in figure skating, though it is incremental as it builds on existing dataset creation efforts.
The authors tackled the lack of comprehensive datasets for artistic sports like figure skating by introducing FSBench, a benchmark dataset that includes multimodal data and QA pairs, revealing significant limitations in existing models' understanding of this domain.
Figure skating, known as the "Art on Ice," is among the most artistic sports, challenging to understand due to its blend of technical elements (like jumps and spins) and overall artistic expression. Existing figure skating datasets mainly focus on single tasks, such as action recognition or scoring, lacking comprehensive annotations for both technical and artistic evaluation. Current sports research is largely centered on ball games, with limited relevance to artistic sports like figure skating. To address this, we introduce FSAnno, a large-scale dataset advancing artistic sports understanding through figure skating. FSAnno includes an open-access training and test dataset, alongside a benchmark dataset, FSBench, for fair model evaluation. FSBench consists of FSBench-Text, with multiple-choice questions and explanations, and FSBench-Motion, containing multimodal data and Question and Answer (QA) pairs, supporting tasks from technical analysis to performance commentary. Initial tests on FSBench reveal significant limitations in existing models' understanding of artistic sports. We hope FSBench will become a key tool for evaluating and enhancing model comprehension of figure skating.