Sufficient and Necessary Explanations (and What Lies in Between)
This work addresses the need for more complete explanations in high-stakes decision-making, though it is incremental in refining existing explanation methods.
The paper tackles the problem of explaining machine learning model predictions by formalizing sufficiency and necessity as precise notions of feature importance, and proposes a unified notion that detects important features missed by previous approaches, showing strong ties to methods like Shapley values.
As complex machine learning models continue to find applications in high-stakes decision-making scenarios, it is crucial that we can explain and understand their predictions. Post-hoc explanation methods provide useful insights by identifying important features in an input $\mathbf{x}$ with respect to the model output $f(\mathbf{x})$. In this work, we formalize and study two precise notions of feature importance for general machine learning models: sufficiency and necessity. We demonstrate how these two types of explanations, albeit intuitive and simple, can fall short in providing a complete picture of which features a model finds important. To this end, we propose a unified notion of importance that circumvents these limitations by exploring a continuum along a necessity-sufficiency axis. Our unified notion, we show, has strong ties to other popular definitions of feature importance, like those based on conditional independence and game-theoretic quantities like Shapley values. Crucially, we demonstrate how a unified perspective allows us to detect important features that could be missed by either of the previous approaches alone.