Efficient Deep Learning of Robust, Adaptive Policies using Tube MPC-Guided Data Augmentation
This work addresses the need for robust and adaptive policies for autonomous systems in unstructured environments, offering an incremental improvement over existing methods by adding adaptation capabilities.
The paper tackled the problem of learning adaptive control policies for agile autonomous systems under uncertainties, by extending an imitation learning algorithm with a learned lower-dimensional representation to enable adaptation, resulting in a policy that achieves a 6.1 cm average position error under wind disturbances 36% larger than training data and trains in about 1.3 hours.
The deployment of agile autonomous systems in challenging, unstructured environments requires adaptation capabilities and robustness to uncertainties. Existing robust and adaptive controllers, such as those based on model predictive control (MPC), can achieve impressive performance at the cost of heavy online onboard computations. Strategies that efficiently learn robust and onboard-deployable policies from MPC have emerged, but they still lack fundamental adaptation capabilities. In this work, we extend an existing efficient Imitation Learning (IL) algorithm for robust policy learning from MPC with the ability to learn policies that adapt to challenging model/environment uncertainties. The key idea of our approach consists in modifying the IL procedure by conditioning the policy on a learned lower-dimensional model/environment representation that can be efficiently estimated online. We tailor our approach to the task of learning an adaptive position and attitude control policy to track trajectories under challenging disturbances on a multirotor. Evaluations in simulation show that a high-quality adaptive policy can be obtained in about $1.3$ hours. We additionally empirically demonstrate rapid adaptation to in- and out-of-training-distribution uncertainties, achieving a $6.1$ cm average position error under wind disturbances that correspond to about $50\%$ of the weight of the robot, and that are $36\%$ larger than the maximum wind seen during training.