Designing Inherently Interpretable Machine Learning Models
This work provides a practical guide for developing inherently interpretable models in high-risk applications like banking, addressing regulatory scrutiny, but it is incremental as it builds on existing methods.
The authors tackled the need for interpretable machine learning in regulated industries by proposing a qualitative template for assessing inherent interpretability based on feature effects and model architecture constraints, with examples from tools like ExNN and a case study on credit default prediction.
Interpretable machine learning (IML) becomes increasingly important in highly regulated industry sectors related to the health and safety or fundamental rights of human beings. In general, the inherently IML models should be adopted because of their transparency and explainability, while black-box models with model-agnostic explainability can be more difficult to defend under regulatory scrutiny. For assessing inherent interpretability of a machine learning model, we propose a qualitative template based on feature effects and model architecture constraints. It provides the design principles for high-performance IML model development, with examples given by reviewing our recent works on ExNN, GAMI-Net, SIMTree, and the Aletheia toolkit for local linear interpretability of deep ReLU networks. We further demonstrate how to design an interpretable ReLU DNN model with evaluation of conceptual soundness for a real case study of predicting credit default in home lending. We hope that this work will provide a practical guide of developing inherently IML models in high risk applications in banking industry, as well as other sectors.