Fault-Tolerant Control of Degrading Systems with On-Policy Reinforcement Learning
This addresses fault tolerance in critical systems like aircraft fuel transfer, offering a novel approach that eliminates the need for fault detection and diagnosis, though it appears incremental in its adaptation of existing RL techniques.
The paper tackles fault-tolerant control of degrading systems without requiring prior fault knowledge or a separate fault detection step, achieving stable learning and improved sample efficiency through an adaptive on-policy reinforcement learning method that combines online and offline learning.
We propose a novel adaptive reinforcement learning control approach for fault tolerant control of degrading systems that is not preceded by a fault detection and diagnosis step. Therefore, \textit{a priori} knowledge of faults that may occur in the system is not required. The adaptive scheme combines online and offline learning of the on-policy control method to improve exploration and sample efficiency, while guaranteeing stable learning. The offline learning phase is performed using a data-driven model of the system, which is frequently updated to track the system's operating conditions. We conduct experiments on an aircraft fuel transfer system to demonstrate the effectiveness of our approach.