SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding
This work addresses the need for more interactive and explainable AI-driven sports analysis for soccer analysts, though it is incremental as it builds on existing multimodal integration methods.
The paper tackled the problem of limited context in soccer video understanding by introducing SoccerChat, a multimodal conversational AI framework that integrates visual and textual data, achieving competitive accuracy in referee decision-making tasks.
The integration of artificial intelligence in sports analytics has transformed soccer video understanding, enabling real-time, automated insights into complex game dynamics. Traditional approaches rely on isolated data streams, limiting their effectiveness in capturing the full context of a match. To address this, we introduce SoccerChat, a multimodal conversational AI framework that integrates visual and textual data for enhanced soccer video comprehension. Leveraging the extensive SoccerNet dataset, enriched with jersey color annotations and automatic speech recognition (ASR) transcripts, SoccerChat is fine-tuned on a structured video instruction dataset to facilitate accurate game understanding, event classification, and referee decision making. We benchmark SoccerChat on action classification and referee decision-making tasks, demonstrating its performance in general soccer event comprehension while maintaining competitive accuracy in referee decision making. Our findings highlight the importance of multimodal integration in advancing soccer analytics, paving the way for more interactive and explainable AI-driven sports analysis. https://github.com/simula/SoccerChat