Control Design for Markov Chains under Safety Constraints: A Convex Approach
It provides a new convex approach for safety-constrained control in Markov chains, but the problem is well-studied and the method appears incremental.
The paper proposes a convex optimization method to design control policies for Markov chains that maximize the set of safe recurrent states under safety constraints, demonstrated with a numerical example.
This paper focuses on the design of time-invariant memoryless control policies for fully observed controlled Markov chains, with a finite state space. Safety constraints are imposed through a pre-selected set of forbidden states. A state is qualified as safe if it is not a forbidden state and the probability of it transitioning to a forbidden state is zero. The main objective is to obtain control policies whose closed loop generates the maximal set of safe recurrent states, which may include multiple recurrent classes. A design method is proposed that relies on a finitely parametrized convex program inspired on entropy maximization principles. A numerical example is provided and the adoption of additional constraints is discussed.