World Models as Reference Trajectories for Rapid Motor Adaptation
This work addresses the problem of performance degradation in real-world robotics and control systems due to dynamic changes, offering a solution for maintaining control in high-dimensional continuous tasks, though it appears incremental as it builds on existing model-based RL and control methods.
The paper tackles the challenge of deploying learned control policies in real-world environments with unexpected dynamic changes by introducing Reflexive World Models (RWM), a dual control framework that uses world model predictions as reference trajectories for rapid adaptation, achieving significantly faster adaptation with low online computational cost compared to baselines while maintaining near-optimal performance.
Deploying learned control policies in real-world environments poses a fundamental challenge. When system dynamics change unexpectedly, performance degrades until models are retrained on new data. We introduce Reflexive World Models (RWM), a dual control framework that uses world model predictions as implicit reference trajectories for rapid adaptation. Our method separates the control problem into long-term reward maximization through reinforcement learning and robust motor execution through rapid latent control. This dual architecture achieves significantly faster adaptation with low online computational cost compared to model-based RL baselines, while maintaining near-optimal performance. The approach combines the benefits of flexible policy learning through reinforcement learning with rapid error correction capabilities, providing a principled approach to maintaining performance in high-dimensional continuous control tasks under varying dynamics.