Adapt Your Body: Mitigating Proprioception Shifts in Imitation Learning
This work addresses a specific bottleneck in robotic imitation learning by mitigating distributional shifts in proprioceptive data, offering an incremental improvement for more robust policy deployment.
The paper tackles the proprioception shift problem in imitation learning, where incorporating proprioceptive states degrades performance due to distributional differences between training and deployment, and proposes a domain adaptation framework that uses Wasserstein distance and noise addition to align distributions, resulting in improved performance over baselines in robotic manipulation tasks.
Imitation learning models for robotic tasks typically rely on multi-modal inputs, such as RGB images, language, and proprioceptive states. While proprioception is intuitively important for decision-making and obstacle avoidance, simply incorporating all proprioceptive states leads to a surprising degradation in imitation learning performance. In this work, we identify the underlying issue as the proprioception shift problem, where the distributions of proprioceptive states diverge significantly between training and deployment. To address this challenge, we propose a domain adaptation framework that bridges the gap by utilizing rollout data collected during deployment. Using Wasserstein distance, we quantify the discrepancy between expert and rollout proprioceptive states and minimize this gap by adding noise to both sets of states, proportional to the Wasserstein distance. This strategy enhances robustness against proprioception shifts by aligning the training and deployment distributions. Experiments on robotic manipulation tasks demonstrate the efficacy of our method, enabling the imitation policy to leverage proprioception while mitigating its adverse effects. Our approach outperforms the naive solution which discards proprioception, and other baselines designed to address distributional shifts.