Explaining Process Control Optimisation Recommendations via GradientSHAP and Implicit Differentiation
For operators of industrial processes, this provides fast, interpretable explanations of optimisation recommendations, potentially increasing trust and adoption.
The paper addresses the trust gap in automated industrial process optimisation by developing an explainable AI method that combines Implicit Function Theorem-based sensitivity analysis with SHAP attribution and LLM-generated narratives. For an HPGR control problem with 22 features, it achieves >0.99 correlation with KernelSHAP and over 40x speedup, enabling real-time explanations.
Automated optimisation is increasingly adopted in industrial processes, yet a trust gap persists between engineers who design these algorithms and operators who must act on their recommendations. Explainable AI methods like SHAP (SHapley Additive exPlanations) have transformed interpretability for machine learning predictions; optimisation outputs could benefit from similar techniques. We present an approach that integrates Implicit Function Theorem (IFT) based sensitivity analysis with SHAP attribution and narrative generation via Large Language Models (LLM), producing explanations tailored for operators. Our approach leverages IFT to compute exact parameter sensitivities $\partial p^*/\partial x$ from the optimality conditions, enabling efficient GradientSHAP computation. For an industrial High Pressure Grinding Roll (HPGR) control optimisation problem with 22 features, we achieve equivalent SHAP attributions (correlation $>$0.99 with KernelSHAP) with over 40$\times$ speedup, enabling real-time natural language explanations. We validate on industrial scenarios and present feedback from domain experts on generated explanations.