UDUC: An Uncertainty-driven Approach for Learning-based Robust Control
This addresses robustness issues in learning-based control for applications like robotics, but it is incremental as it builds on existing probabilistic ensemble methods.
The paper tackles the problem of mode collapse in probabilistic ensemble models for robust control, introducing the UDUC loss to improve robustness against environmental mismatches, achieving performance evaluated on the RWRL benchmark.
Learning-based techniques have become popular in both model predictive control (MPC) and reinforcement learning (RL). Probabilistic ensemble (PE) models offer a promising approach for modelling system dynamics, showcasing the ability to capture uncertainty and scalability in high-dimensional control scenarios. However, PE models are susceptible to mode collapse, resulting in non-robust control when faced with environments slightly different from the training set. In this paper, we introduce the $\textbf{u}$ncertainty-$\textbf{d}$riven rob$\textbf{u}$st $\textbf{c}$ontrol (UDUC) loss as an alternative objective for training PE models, drawing inspiration from contrastive learning. We analyze the robustness of UDUC loss through the lens of robust optimization and evaluate its performance on the challenging Real-world Reinforcement Learning (RWRL) benchmark, which involves significant environmental mismatches between the training and testing environments.