MultiVENT: Multilingual Videos of Events with Aligned Natural Text
This dataset addresses the need for multimodal, multilingual resources to improve model robustness in news analysis, though it is incremental as it builds on existing video-text datasets.
The authors tackled the lack of diverse multilingual news video datasets by constructing MultiVENT, which includes videos and aligned text in five languages, and they provided a baseline model for multilingual video retrieval.
Everyday news coverage has shifted from traditional broadcasts towards a wide range of presentation formats such as first-hand, unedited video footage. Datasets that reflect the diverse array of multimodal, multilingual news sources available online could be used to teach models to benefit from this shift, but existing news video datasets focus on traditional news broadcasts produced for English-speaking audiences. We address this limitation by constructing MultiVENT, a dataset of multilingual, event-centric videos grounded in text documents across five target languages. MultiVENT includes both news broadcast videos and non-professional event footage, which we use to analyze the state of online news videos and how they can be leveraged to build robust, factually accurate models. Finally, we provide a model for complex, multilingual video retrieval to serve as a baseline for information retrieval using MultiVENT.