Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models
This work addresses the lack of robust benchmarks and high-quality data for video understanding reward modeling, which is a bottleneck for progress in this domain.
The paper introduces VURB, a benchmark with 2,100 preference pairs for video understanding reward modeling, and VUP-35K, a large-scale preference dataset. The trained reward models VideoDRM and VideoGRM achieve state-of-the-art performance on VURB and VideoRewardBench, and improve test-time scaling.
Multimodal reward models have advanced substantially in text and image domains, yet progress in video understanding reward modeling remains severely limited by the lack of robust evaluation benchmarks and high-quality preference data. To address this, we propose a unified framework spanning benchmark design, data construction, and reward model training. We introduce Video Understanding Reward Bench (VURB), a benchmark featuring 2,100 preference pairs with long chain-of-thought reasoning traces (averaging 1,143 tokens) and majority voting evaluation across general, long, and reasoning-oriented video tasks. We further construct Video Understanding Preference Dataset (VUP-35K) via a fully automated pipeline, providing large-scale high-quality supervision for video reward training. Building on the data, we train VideoDRM and VideoGRM, a discriminative and a generative reward model, both achieving state-of-the-art performance on VURB and VideoRewardBench. Further analysis confirms that VUP-35K enhances both reward performance and model reasoning capability, while VideoDRM and VideoGRM yield significant gains under best-of-$N$ test-time scaling.