SHAP-Based Supervised Clustering for Sample Classification and the Generalized Waterfall Plot
This addresses the need for interpretability in high-stakes applications like healthcare, but is incremental as it builds on existing SHAP methods.
The paper tackles the problem of explaining black-box model predictions by clustering SHAP values to group samples with similar prediction reasons, and demonstrates this with a simulated experiment and an Alzheimer's disease case study, achieving insights into distinct pathways for predictions.
In this growing age of data and technology, large black-box models are becoming the norm due to their ability to handle vast amounts of data and learn incredibly complex input-output relationships. The deficiency of these methods, however, is their inability to explain the prediction process, making them untrustworthy and their use precarious in high-stakes situations. SHapley Additive exPlanations (SHAP) analysis is an explainable AI method growing in popularity for its ability to explain model predictions in terms of the original features. For each sample and feature in the data set, we associate a SHAP value that quantifies the contribution of that feature to the prediction of that sample. Clustering these SHAP values can provide insight into the data by grouping samples that not only received the same prediction, but received the same prediction for similar reasons. In doing so, we map the various pathways through which distinct samples arrive at the same prediction. To showcase this methodology, we present a simulated experiment in addition to a case study in Alzheimer's disease using data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. We also present a novel generalization of the waterfall plot for multi-classification.