Eduardo Arvelo

Eduardo Arvelo, Nuno C. Martins

This paper focuses on the design of time-invariant memoryless control policies for fully observed controlled Markov chains, with a finite state space. Safety constraints are imposed through a pre-selected set of forbidden states. A state is qualified as safe if it is not a forbidden state and the probability of it transitioning to a forbidden state is zero. The main objective is to obtain control policies whose closed loop generates the maximal set of safe recurrent states, which may include multiple recurrent classes. A design method is proposed that relies on a finitely parametrized convex program inspired on entropy maximization principles. A numerical example is provided and the adoption of additional constraints is discussed.

Eduardo Arvelo, Nuno C. Martins

We propose a receding horizon control strategy that readily handles systems that exhibit interval-wise total energy constraints on the input control sequence. The approach is based on a variable optimization horizon length and contractive final state constraint sets. The optimization horizon, which recedes by N steps every N steps, is the key to accommodate the interval-wise total energy constraints. The varying optimization horizon along with the contractive constraints are used to achieve analytic asymptotic stability of the system under the proposed scheme. The strategy is demonstrated by simulation examples.

Eduardo Arvelo, Eric Kim, Nuno C. Martins

This paper deals with the design of time-invariant memoryless control policies for robots that move in a finite two- dimensional lattice and are tasked with persistent surveillance of an area in which there are forbidden regions. We model each robot as a controlled Markov chain whose state comprises its position in the lattice and the direction of motion. The goal is to find the minimum number of robots and an associated time-invariant memoryless control policy that guarantees that the largest number of states are persistently surveilled without ever visiting a forbidden state. We propose a design method that relies on a finitely parametrized convex program inspired by entropy maximization principles. Numerical examples are provided.