Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark
This work addresses control performance in non-linear stochastic systems, specifically for throttle valves, but is incremental as it builds on existing PI and RL methods.
The paper tackled improving control of non-linear throttle valves with asymmetric hysteresis by combining a Proportional Integral (PI) controller with Reinforcement Learning (RL), resulting in a near-optimal controller that outperforms the PI controller and has better sample efficiency than traditional RL agents in experimental tests on three valves.
This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric hysteresis, leading to a near-optimal controller without requiring any prior knowledge about the environment. We start with a carefully tuned Proportional Integrator (PI) controller and exploit the recent advances in Reinforcement Learning (RL) with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. We test the proposed control method in various scenarios on three different valves, all highlighting the benefits of combining both PI and RL frameworks to improve control performance in non-linear stochastic systems. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller.