Towards Universal Soccer Video Understanding
This work addresses the need for advanced video analysis in soccer, a popular global sport, by providing a new dataset and model that improve performance on specific tasks, though it is incremental in building on existing multi-modal frameworks.
The paper tackles the problem of comprehensive soccer video understanding by introducing SoccerReplay-1988, the largest multi-modal soccer dataset with automated annotations, and MatchVision, a soccer-specific visual encoder that achieves state-of-the-art performance on tasks like event classification and commentary generation.
As a globally celebrated sport, soccer has attracted widespread interest from fans all over the world. This paper aims to develop a comprehensive multi-modal framework for soccer video understanding. Specifically, we make the following contributions in this paper: (i) we introduce SoccerReplay-1988, the largest multi-modal soccer dataset to date, featuring videos and detailed annotations from 1,988 complete matches, with an automated annotation pipeline; (ii) we present an advanced soccer-specific visual encoder, MatchVision, which leverages spatiotemporal information across soccer videos and excels in various downstream tasks; (iii) we conduct extensive experiments and ablation studies on event classification, commentary generation, and multi-view foul recognition. MatchVision demonstrates state-of-the-art performance on all of them, substantially outperforming existing models, which highlights the superiority of our proposed data and model. We believe that this work will offer a standard paradigm for sports understanding research.