SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World Models
For embodied AI and robotics, SKIP addresses the computational bottleneck of long-horizon video generation while preserving task-critical events, enabling efficient policy training.
SKIP proposes a sparse-to-dense framework that generates only task-relevant keyframes and interpolates missing frames, achieving 4.16x faster rollout generation while reducing FVD by 89.0% on LIBERO. When used as training data, policy success drops only 1.3 pp in simulation and 6.7 pp on real robots, compared to 48-58 pp drops with dense generation.
Embodied world models have emerged as a promising paradigm in robotics by predicting how robot actions affect the surrounding scene. However, the rollout inference remains computationally expensive in pixel space, as long-horizon manipulation videos typically have to be generated frame by frame. This cost cannot be easily reduced by indiscriminately dropping frames, since downstream policies rely on complete preservation of sparse task-relevant events such as approach, contact, grasp, and release. To address this challenge, we propose Sparse Keyframe Interpolation Paradigm (SKIP), an event-preserving sparse-to-dense framework that avoids dense frame-by-frame generation. SKIP first identifies task-relevant keyframes by leveraging robot-aware multimodal features. It then synthesizes only these keyframes with a sparse video diffusion model. A learned gap predictor and an action-conditioned interpolator subsequently reconstruct the missing intervals according to the robot actions. On LIBERO, SKIP generates dense rollouts $4.16\times$ faster than a dense baseline while improving visual fidelity and reducing aggregate FVD by $89.0\%$. Importantly, SKIP-generated videos are effective policy-training data. Even when they fully replace real demonstrations, $π_{0.5}$ success drops only $1.3$ pp in LIBERO simulation and $6.7$ pp on the real robot, whereas fully dense frame-by-frame generation collapses by $48$ to $58$ pp.