Causal Proxy Models for Concept-Based Model Explanations
This addresses the challenge of creating explainable AI for NLP practitioners, offering a method to replace black-box models with more interpretable ones, though it is incremental in leveraging existing counterfactual approximations.
The paper tackles the problem of causal explainability in NLP systems by introducing Causal Proxy Models (CPMs), which approximate counterfactuals to simulate black-box model behavior, resulting in models that perform comparably in factual predictions while enabling explainability.
Explainability methods for NLP systems encounter a version of the fundamental problem of causal inference: for a given ground-truth input text, we never truly observe the counterfactual texts necessary for isolating the causal effects of model representations on outputs. In response, many explainability methods make no use of counterfactual texts, assuming they will be unavailable. In this paper, we show that robust causal explainability methods can be created using approximate counterfactuals, which can be written by humans to approximate a specific counterfactual or simply sampled using metadata-guided heuristics. The core of our proposal is the Causal Proxy Model (CPM). A CPM explains a black-box model $\mathcal{N}$ because it is trained to have the same actual input/output behavior as $\mathcal{N}$ while creating neural representations that can be intervened upon to simulate the counterfactual input/output behavior of $\mathcal{N}$. Furthermore, we show that the best CPM for $\mathcal{N}$ performs comparably to $\mathcal{N}$ in making factual predictions, which means that the CPM can simply replace $\mathcal{N}$, leading to more explainable deployed models. Our code is available at https://github.com/frankaging/Causal-Proxy-Model.