Robust Behavior Cloning Via Global Lipschitz Regularization
This work addresses robustness in imitation learning for safety-critical domains like autonomous vehicles, but it is incremental as it builds on existing Lipschitz regularization methods.
The paper tackles the problem of behavior cloning policies being vulnerable to measurement errors or adversarial disturbances during deployment by using global Lipschitz regularization to enhance robustness, and empirically validates this approach across various Gymnasium environments.
Behavior Cloning (BC) is an effective imitation learning technique and has even been adopted in some safety-critical domains such as autonomous vehicles. BC trains a policy to mimic the behavior of an expert by using a dataset composed of only state-action pairs demonstrated by the expert, without any additional interaction with the environment. However, During deployment, the policy observations may contain measurement errors or adversarial disturbances. Since the observations may deviate from the true states, they can mislead the agent into making sub-optimal actions. In this work, we use a global Lipschitz regularization approach to enhance the robustness of the learned policy network. We then show that the resulting global Lipschitz property provides a robustness certificate to the policy with respect to different bounded norm perturbations. Then, we propose a way to construct a Lipschitz neural network that ensures the policy robustness. We empirically validate our theory across various environments in Gymnasium. Keywords: Robust Reinforcement Learning; Behavior Cloning; Lipschitz Neural Network