LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE
This addresses a bottleneck in generating high-quality embodied manipulation data for robotics, representing a novel method for a known issue.
The paper tackled the problem of stable long-horizon video generation for embodied manipulation by introducing LongScape, a hybrid framework that combines diffusion and autoregressive methods with action-guided chunking and a context-aware Mixture-of-Experts, achieving stable and consistent generation over extended rollouts.
Video-based world models hold significant potential for generating high-quality embodied manipulation data. However, current video generation methods struggle to achieve stable long-horizon generation: classical diffusion-based approaches often suffer from temporal inconsistency and visual drift over multiple rollouts, while autoregressive methods tend to compromise on visual detail. To solve this, we introduce LongScape, a hybrid framework that adaptively combines intra-chunk diffusion denoising with inter-chunk autoregressive causal generation. Our core innovation is an action-guided, variable-length chunking mechanism that partitions video based on the semantic context of robotic actions. This ensures each chunk represents a complete, coherent action, enabling the model to flexibly generate diverse dynamics. We further introduce a Context-aware Mixture-of-Experts (CMoE) framework that adaptively activates specialized experts for each chunk during generation, guaranteeing high visual quality and seamless chunk transitions. Extensive experimental results demonstrate that our method achieves stable and consistent long-horizon generation over extended rollouts. Our code is available at: https://github.com/tsinghua-fib-lab/Longscape.