Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning
This work addresses safety-critical control problems in reinforcement learning, but it appears incremental as it builds on existing model-based and uncertainty-aware methods.
The paper tackled the problem of managing risk in model-based reinforcement learning by introducing a method that separates epistemic and aleatoric uncertainties, using probabilistic safety constraints and an ensemble of stochastic neural networks. The result showed that this separation is essential for good performance in uncertain and safety-critical control environments, as indicated by various experiments.
We introduce a simple but effective method for managing risk in model-based reinforcement learning with trajectory sampling that involves probabilistic safety constraints and balancing of optimism in the face of epistemic uncertainty and pessimism in the face of aleatoric uncertainty of an ensemble of stochastic neural networks.Various experiments indicate that the separation of uncertainties is essential to performing well with data-driven MPC approaches in uncertain and safety-critical control environments.