Information-Theoretic Safe Exploration with Gaussian Processes
This addresses safe exploration for decision-making systems where safety constraints are unknown, but it is incremental as it builds on existing GP-based methods.
The paper tackles the problem of safe exploration in sequential decision-making with unknown safety constraints by proposing an information-theoretic criterion using Gaussian processes, which avoids discretization and hyperparameters, and empirically shows improved data-efficiency and scalability.
We consider a sequential decision making task where we are not allowed to evaluate parameters that violate an a priori unknown (safety) constraint. A common approach is to place a Gaussian process prior on the unknown constraint and allow evaluations only in regions that are safe with high probability. Most current methods rely on a discretization of the domain and cannot be directly extended to the continuous case. Moreover, the way in which they exploit regularity assumptions about the constraint introduces an additional critical hyperparameter. In this paper, we propose an information-theoretic safe exploration criterion that directly exploits the GP posterior to identify the most informative safe parameters to evaluate. Our approach is naturally applicable to continuous domains and does not require additional hyperparameters. We theoretically analyze the method and show that we do not violate the safety constraint with high probability and that we explore by learning about the constraint up to arbitrary precision. Empirical evaluations demonstrate improved data-efficiency and scalability.