Linking Model Intervention to Causal Interpretation in Model Explanation
This work addresses the trustworthiness of ML models for domain experts by clarifying when explanation methods provide causal insights, though it is incremental in linking existing intervention concepts to causal interpretation.
The paper investigates when model intervention effects in machine learning explanations can be interpreted causally, specifically linking them to direct causation of features on outcomes, and validates the conditions using semi-synthetic datasets.
Intervention intuition is often used in model explanation where the intervention effect of a feature on the outcome is quantified by the difference of a model prediction when the feature value is changed from the current value to the baseline value. Such a model intervention effect of a feature is inherently association. In this paper, we will study the conditions when an intuitive model intervention effect has a causal interpretation, i.e., when it indicates whether a feature is a direct cause of the outcome. This work links the model intervention effect to the causal interpretation of a model. Such an interpretation capability is important since it indicates whether a machine learning model is trustworthy to domain experts. The conditions also reveal the limitations of using a model intervention effect for causal interpretation in an environment with unobserved features. Experiments on semi-synthetic datasets have been conducted to validate theorems and show the potential for using the model intervention effect for model interpretation.