Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts
This work addresses the challenge of distribution shifts in robotics for improved safety and adaptability, representing a novel method for a known bottleneck.
The paper tackles the problem of enabling robots to detect and react to distribution shifts in sensorimotor control by introducing conformal policy learning, which uses conformal quantiles to switch between policies with formal statistical guarantees, and it outperforms five baselines in simulated autonomous driving and physical quadruped experiments.
This paper focuses on the problem of detecting and reacting to changes in the distribution of a sensorimotor controller's observables. The key idea is the design of switching policies that can take conformal quantiles as input, which we define as conformal policy learning, that allows robots to detect distribution shifts with formal statistical guarantees. We show how to design such policies by using conformal quantiles to switch between base policies with different characteristics, e.g. safety or speed, or directly augmenting a policy observation with a quantile and training it with reinforcement learning. Theoretically, we show that such policies achieve the formal convergence guarantees in finite time. In addition, we thoroughly evaluate their advantages and limitations on two compelling use cases: simulated autonomous driving and active perception with a physical quadruped. Empirical results demonstrate that our approach outperforms five baselines. It is also the simplest of the baseline strategies besides one ablation. Being easy to use, flexible, and with formal guarantees, our work demonstrates how conformal prediction can be an effective tool for sensorimotor learning under uncertainty.