Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control
This work addresses uncertainty estimation for reinforcement learning agents in real-world applications like robotics and power grids, but it is incremental as it extends an existing method.
The authors tackled the challenge of uncertainty quantification in reinforcement learning by proposing an uncertainty-aware algorithm for continuous control tasks, which outperformed vanilla DDPG in robotic control and power-grid optimization benchmarks.
Uncertainty quantification is one of the central challenges for machine learning in real-world applications. In reinforcement learning, an agent confronts two kinds of uncertainty, called epistemic uncertainty and aleatoric uncertainty. Disentangling and evaluating these uncertainties simultaneously stands a chance of improving the agent's final performance, accelerating training, and facilitating quality assurance after deployment. In this work, we propose an uncertainty-aware reinforcement learning algorithm for continuous control tasks that extends the Deep Deterministic Policy Gradient algorithm (DDPG). It exploits epistemic uncertainty to accelerate exploration and aleatoric uncertainty to learn a risk-sensitive policy. We conduct numerical experiments showing that our variant of DDPG outperforms vanilla DDPG without uncertainty estimation in benchmark tasks on robotic control and power-grid optimization.