UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
This addresses the challenge of scalable and efficient robot policy learning for robotics, though it appears incremental as it builds on existing VLA methods with a novel latent action approach.
The paper tackles the problem of generalist robots struggling to transfer knowledge across different embodiments and environments by proposing UniVLA, a framework that learns cross-embodiment vision-language-action policies using task-centric latent actions from videos, achieving state-of-the-art results on multiple benchmarks with superior performance over OpenVLA using less than 1/20 of pretraining compute and 1/10 of downstream data.
A generalist robot should perform effectively across various environments. However, most existing approaches heavily rely on scaling action-annotated data to enhance their capabilities. Consequently, they are often limited to single physical specification and struggle to learn transferable knowledge across different embodiments and environments. To confront these limitations, we propose UniVLA, a new framework for learning cross-embodiment vision-language-action (VLA) policies. Our key innovation is to derive task-centric action representations from videos with a latent action model. This enables us to exploit extensive data across a wide spectrum of embodiments and perspectives. To mitigate the effect of task-irrelevant dynamics, we incorporate language instructions and establish a latent action model within the DINO feature space. Learned from internet-scale videos, the generalist policy can be deployed to various robots through efficient latent action decoding. We obtain state-of-the-art results across multiple manipulation and navigation benchmarks, as well as real-robot deployments. UniVLA achieves superior performance over OpenVLA with less than 1/20 of pretraining compute and 1/10 of downstream data. Continuous performance improvements are observed as heterogeneous data, even including human videos, are incorporated into the training pipeline. The results underscore UniVLA's potential to facilitate scalable and efficient robot policy learning.