DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning
It addresses the problem of distribution shift during deployment for pretrained VLA policies, offering a finetuning-free solution that enhances robustness and instruction following.
DREAMSTEER improves deployment-time robustness of pretrained VLA policies without finetuning by using a latent world model and value model to steer action selection, achieving a 42.5% absolute increase in task success rate (from 23.75% to 66.25%) and a 17.5% increase in instruction-following accuracy (from 38.75% to 56.25%) across four real-world manipulation benchmarks.
Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness and inconsistent instruction following. While prior work commonly tackles this by finetuning on in-distribution data, it assumes demonstrations collected on tasks in the target environment. In this work, we propose DREAMSTEER, a deployment-time steering framework for pretrained VLAs without any finetuning or parameter modifications. The key insight in DREAMSTEER is to leverage a latent world model and a value model to steer pretrained VLA policies. During deployment, DREAMSTEER samples candidate action chunks from a VLA policy and predefined motion primitives, imagines their outcomes using an action-conditioned latent world model, and ranks the imagined trajectories with a language-conditioned value model. Across four real-world manipulation benchmarks with unseen objects, DREAMSTEER improves task success rate from 23.75% to 66.25% and instruction-following accuracy from 38.75% to 56.25% over the base VLA policy.