Improving Model-Based Control and Active Exploration with Reconstruction Uncertainty Optimization
This work addresses computational inefficiency and unreliability in model-based control and exploration for dynamical systems, representing an incremental improvement through a novel uncertainty estimation technique.
The paper tackles the problem of inaccurate predictions in model-based control by proposing a model-agnostic method to estimate uncertainty based on reconstruction error, which improves control performance across various environments and enables more efficient active exploration for faster model learning.
Model based predictions of future trajectories of a dynamical system often suffer from inaccuracies, forcing model based control algorithms to re-plan often, thus being computationally expensive, suboptimal and not reliable. In this work, we propose a model agnostic method for estimating the uncertainty of a model?s predictions based on reconstruction error, using it in control and exploration. As our experiments show, this uncertainty estimation can be used to improve control performance on a wide variety of environments by choosing predictions of which the model is confident. It can also be used for active learning to explore more efficiently the environment by planning for trajectories with high uncertainty, allowing faster model learning.