On the Optimality of Uncertain MDP Abstractions
Provides theoretical foundations for abstraction-based control synthesis, clarifying which abstraction types guarantee optimality for nonlinear stochastic systems with temporal logic specifications.
This paper investigates whether refinement of uncertain MDP (UMDP) abstractions leads to optimal control synthesis for nonlinear stochastic systems. It identifies a sufficient condition called vanishing ambiguity that guarantees asymptotic optimality and algorithm completeness, showing set-valued MDP abstractions satisfy this while interval MDP abstractions do not.
We study the asymptotic optimality of abstraction-based control synthesis algorithms. Specifically, we consider uncertain MDP (UMDP) abstraction, and investigate whether refinement leads to optimal results, i.e., an optimal controller and zero error bound. Additionally, we study completeness of abstraction-refinement algorithms, i.e., that the algorithm produces near-optimal results in finite time. The focus is on nonlinear stochastic systems with general vector fields and temporal logic specifications. We present an algorithm that abstracts the system into a UMDP and synthesizes a controller with performance guarantees via robust dynamic programming. Then, the algorithm iteratively refines the abstraction until a near-optimality criterion is met. A thorough theoretical analysis reveals a sufficient condition, which we denote vanishing ambiguity, guaranteeing asymptotic optimality of the abstraction process and completeness of the algorithm. We show that set-valued MDP abstractions satisfy this criterion, whereas interval MDP abstractions lack such a guarantee.