Safe and Adaptive Decision-Making for Optimization of Safety-Critical Systems: The ARTEO Algorithm
This work addresses safety-critical optimization for business and industrial applications, offering an incremental improvement over existing safe bandit methods.
The authors tackled the problem of decision-making under uncertainty with safety constraints in real-time optimization, proposing the ARTEO algorithm that uses Gaussian processes to quantify uncertainty and adaptively control exploration, achieving less cumulative regret and safe decisions in case studies like electric motor current optimization and real-time bidding.
We consider the problem of decision-making under uncertainty in an environment with safety constraints. Many business and industrial applications rely on real-time optimization to improve key performance indicators. In the case of unknown characteristics, real-time optimization becomes challenging, particularly because of the satisfaction of safety constraints. We propose the ARTEO algorithm, where we cast multi-armed bandits as a mathematical programming problem subject to safety constraints and learn the unknown characteristics through exploration while optimizing the targets. We quantify the uncertainty in unknown characteristics by using Gaussian processes and incorporate it into the cost function as a contribution which drives exploration. We adaptively control the size of this contribution in accordance with the requirements of the environment. We guarantee the safety of our algorithm with a high probability through confidence bounds constructed under the regularity assumptions of Gaussian processes. We demonstrate the safety and efficiency of our approach with two case studies: optimization of electric motor current and real-time bidding problems. We further evaluate the performance of ARTEO compared to a safe variant of upper confidence bound based algorithms. ARTEO achieves less cumulative regret with accurate and safe decisions.