VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition
This addresses the challenge of robust video post-training for VideoLLMs, offering a scalable solution, though it is incremental as it builds on existing curriculum learning methods.
The paper tackles the problem of inefficient curriculum strategies in reinforcement learning for video understanding by proposing VideoCuRL, which decomposes difficulty into orthogonal axes and uses a 2D curriculum grid, resulting in improvements of +2.5 on reasoning and +2.9 on perception tasks.
Reinforcement Learning (RL) is crucial for empowering VideoLLMs with complex spatiotemporal reasoning. However, current RL paradigms predominantly rely on random data shuffling or naive curriculum strategies based on scalar difficulty metrics. We argue that scalar metrics fail to disentangle two orthogonal challenges in video understanding: Visual Temporal Perception Load and Cognitive Reasoning Depth. To address this, we propose VideoCuRL, a novel framework that decomposes difficulty into these two axes. We employ efficient, training-free proxies, optical flow and keyframe entropy for visual complexity, Calibrated Surprisal for cognitive complexity, to map data onto a 2D curriculum grid. A competence aware Diagonal Wavefront strategy then schedules training from base alignment to complex reasoning. Furthermore, we introduce Dynamic Sparse KL and Structured Revisiting to stabilize training against reward collapse and catastrophic forgetting. Extensive experiments show that VideoCuRL surpasses strong RL baselines on reasoning (+2.5 on VSI-Bench) and perception (+2.9 on VideoMME) tasks. Notably, VideoCuRL eliminates the prohibitive inference overhead of generation-based curricula, offering a scalable solution for robust video post-training.