Linear quadratic regulation of polytopic time-inhomogeneous Markov jump linear systems (extended version)
For control theorists and engineers, this extends LQR theory to a more realistic class of systems with time-varying and uncertain transition probabilities.
The paper solves the infinite-horizon optimal control problem for Markov jump linear systems with polytopic time-inhomogeneous transition probabilities, showing that the optimal controller is obtained from coupled algebraic Riccati equations and that the finite-horizon cost converges exponentially to the infinite-horizon cost.
In most real cases transition probabilities between operational modes of Markov jump linear systems cannot be computed exactly and are time-varying. We take into account this aspect by considering Markov jump linear systems where the underlying Markov chain is polytopic and time-inhomogeneous, i.e. its transition probability matrix is varying over time, with variations that are arbitrary within a polytopic set of stochastic matrices. We address and solve for this class of systems the infinite-horizon optimal control problem. In particular, we show that the optimal controller can be obtained from a set of coupled algebraic Riccati equations, and that for mean square stabilizable systems the optimal finite-horizon cost corresponding to the solution to a parsimonious set of coupled difference Riccati equations converges exponentially fast to the optimal infinite-horizon cost related to the set of coupled algebraic Riccati equations. All the presented concepts are illustrated on a numerical example showing the efficiency of the provided solution.