STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation
For robotic manipulation, STARRY provides a novel action-centric spatial-temporal world modeling approach that significantly improves performance in spatially and temporally demanding tasks.
STARRY introduces a world-model-enhanced policy that jointly denoises future spatial-temporal latents and action sequences, achieving 93.82%/93.30% success on RoboTwin 2.0 and improving real-world success from 42.5% to 70.8% over π0.5.
Robotic manipulation critically requires reasoning about future spatial-temporal interactions, yet existing VLA policies and world-model-enhanced policies do not fully model action-relevant spatial-temporal interaction structure. We propose STARRY, a world-model-enhanced action-generation policy that aligns spatial-temporal prediction with action generation. STARRY jointly denoises future spatial-temporal latents and action sequences, and introduces Geometry-Aware Selective Attention Modulation to convert predicted depth and end-effector geometry into token-aligned weights for selective action-attention modulation. On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings. Real-world experiments further improve average success from 42.5% to 70.8% over $π_{0.5}$, demonstrating the effectiveness of action-centric spatial-temporal world modeling for spatial-temporally demanding robotic action generation.