Multimodal Chaptering for Long-Form TV Newscast Video
This work addresses the problem of structuring and organizing large unsegmented broadcast archives for TV channels, representing an incremental improvement in multimodal video analysis.
The paper tackles automatic chaptering of long TV newscast videos by integrating audio and visual cues with a two-stage neural network approach, achieving state-of-the-art performance with 82% precision at 90% IoU on a dataset of over 500 videos.
We propose a novel approach for automatic chaptering of TV newscast videos, addressing the challenge of structuring and organizing large collections of unsegmented broadcast content. Our method integrates both audio and visual cues through a two-stage process involving frozen neural networks and a trained LSTM network. The first stage extracts essential features from separate modalities, while the LSTM effectively fuses these features to generate accurate segment boundaries. Our proposed model has been evaluated on a diverse dataset comprising over 500 TV newscast videos of an average of 41 minutes gathered from TF1, a French TV channel, with varying lengths and topics. Experimental results demonstrate that this innovative fusion strategy achieves state of the art performance, yielding a high precision rate of 82% at IoU of 90%. Consequently, this approach significantly enhances analysis, indexing and storage capabilities for TV newscast archives, paving the way towards efficient management and utilization of vast audiovisual resources.