Interpretable Hidden Markov Model-Based Deep Reinforcement Learning Hierarchical Framework for Predictive Maintenance of Turbofan Engines
This work addresses predictive maintenance for turbofan engines, providing an interpretable solution that is incremental in combining existing techniques.
The paper tackles the problem of focusing policy learning in sparse domains by combining hidden Markov models with reinforcement learning for predictive maintenance of turbofan engines, resulting in a method that outperforms baseline deep reinforcement learning and hidden Markov model approaches while offering interpretability.
An open research question in deep reinforcement learning is how to focus the policy learning of key decisions within a sparse domain. This paper emphasizes combining the advantages of inputoutput hidden Markov models and reinforcement learning towards interpretable maintenance decisions. We propose a novel hierarchical-modeling methodology that, at a high level, detects and interprets the root cause of a failure as well as the health degradation of the turbofan engine, while, at a low level, it provides the optimal replacement policy. It outperforms the baseline performance of deep reinforcement learning methods applied directly to the raw data or when using a hidden Markov model without such a specialized hierarchy. It also provides comparable performance to prior work, however, with the additional benefit of interpretability.