A reinforcement learning approach to hybrid control design
This work addresses hybrid control design for unknown systems, but it is incremental as it adapts existing RL methods to a specific domain.
The paper tackles the problem of designing hybrid control policies for systems with unknown mathematical models by modeling the problem as a single Markov Decision Process (MDP) and applying an adapted Proximal Policy Optimization (PPO) algorithm, resulting in convergence to optimal policies for benchmark examples.
In this paper we design hybrid control policies for hybrid systems whose mathematical models are unknown. Our contributions are threefold. First, we propose a framework for modelling the hybrid control design problem as a single Markov Decision Process (MDP). This result facilitates the application of off-the-shelf algorithms from Reinforcement Learning (RL) literature towards designing optimal control policies. Second, we model a set of benchmark examples of hybrid control design problem in the proposed MDP framework. Third, we adapt the recently proposed Proximal Policy Optimisation (PPO) algorithm for the hybrid action space and apply it to the above set of problems. It is observed that in each case the algorithm converges and finds the optimal policy.