Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs
This addresses safety guarantees for stochastic control systems (e.g., robotics, autonomous vehicles) but appears incremental as it adapts existing MDP techniques to a specific safety objective.
The paper tackles the problem of ensuring safety in stochastic control systems by developing a new algorithm that reduces safety constraints to average reward Markov Decision Processes, enabling computation of safe policies using standard techniques like linear programs. Results on Double Integrator and Inverted Pendulum systems show the method provides more comprehensive solutions, faster convergence, and higher quality compared to minimum discounted-reward approaches.
Safety in stochastic control systems, which are subject to random noise with a known probability distribution, aims to compute policies that satisfy predefined operational constraints with high confidence throughout the uncertain evolution of the state variables. The unpredictable evolution of state variables poses a significant challenge for meeting predefined constraints using various control methods. To address this, we present a new algorithm that computes safe policies to determine the safety level across a finite state set. This algorithm reduces the safety objective to the standard average reward Markov Decision Process (MDP) objective. This reduction enables us to use standard techniques, such as linear programs, to compute and analyze safe policies. We validate the proposed method numerically on the Double Integrator and the Inverted Pendulum systems. Results indicate that the average-reward MDPs solution is more comprehensive, converges faster, and offers higher quality compared to the minimum discounted-reward solution.