VideoWeave: A Data-Centric Approach for Efficient Video Understanding
This addresses the problem of expensive and data-scarce video understanding for AI researchers, offering an incremental data-centric improvement.
The paper tackles the high cost and limited data for training video-language models by introducing VideoWeave, which constructs synthetic long-context training samples by splicing short, captioned videos, leading to higher accuracy under identical compute constraints compared to conventional methods.
Training video-language models is often prohibitively expensive due to the high cost of processing long frame sequences and the limited availability of annotated long videos. We present VideoWeave, a simple yet effective approach to improve data efficiency by constructing synthetic long-context training samples that splice together short, captioned videos from existing datasets. Rather than modifying model architectures or optimization objectives, VideoWeave reorganizes available video-text pairs to expand temporal diversity within fixed compute. We systematically study how different data composition strategies like random versus visually clustered splicing and caption enrichment affect downstream performance on downstream video question answering. Under identical compute constraints, models trained with VideoWeave achieve higher accuracy than conventional video finetuning. Our results highlight that reorganizing training data, rather than altering architectures, may offer a simple and scalable path for training video-language models. We link our code for all experiments here.