RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models
This work addresses the inefficiency of passive imitation learning for robot policy fine-tuning by introducing active data collection, though the findings are incremental and reveal open challenges.
The paper proposes an active, continual learning paradigm for Vision-Language-Action models, showing that uncertainty-guided data collection leads to more efficient fine-tuning than passive demonstrations, but also causes catastrophic forgetting; they evaluate continual learning techniques and identify tradeoffs between plasticity and retention.
Vision-Language-Action (VLA) models are commonly fine-tuned through passive imitation learning, where additional demonstrations are collected for tasks where the policy performs poorly. This approach incurs several downsides: it requires the robot to fail before data collection is triggered, provides little guidance about which states require supervision, and wastes demonstrator effort on redundant parts of the task where the policy already performs well. In this paper, we propose an active, continual learning paradigm for VLAs. We demonstrate that active, uncertainty-guided data collection leads to more efficient fine-tuning than when using passively-collected demonstrations. However, we also find that fine-tuning only on actively-collected recovery data leads to catastrophic forgetting. We evaluate techniques for continual learning, including replay-based data mixing and elastic weight consolidation, and identify tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors. Overall, our work contributes an empirical study of active continual learning for autoregressive VLAs, establishing that uncertainty-guided recovery demonstrations can improve adaptation efficiency while also revealing open challenges when targeted new data is incorporated into large robot policies.