Global Saliency: Aggregating Saliency Maps to Assess Dataset Artefact Bias
This work addresses the need for interpretability in high-stakes applications like medical imaging to ensure models rely on robust features rather than artefacts, though it is incremental by extending local saliency methods to global aggregation.
The paper tackles the problem of assessing dataset artefact bias in machine learning models by proposing a method to aggregate saliency maps globally using semantic segmentation masks, providing quantitative measures of model bias across a dataset, and applies it to skin lesion diagnosis to evaluate the effect of artefacts like ink.
In high-stakes applications of machine learning models, interpretability methods provide guarantees that models are right for the right reasons. In medical imaging, saliency maps have become the standard tool for determining whether a neural model has learned relevant robust features, rather than artefactual noise. However, saliency maps are limited to local model explanation because they interpret predictions on an image-by-image basis. We propose aggregating saliency globally, using semantic segmentation masks, to provide quantitative measures of model bias across a dataset. To evaluate global saliency methods, we propose two metrics for quantifying the validity of saliency explanations. We apply the global saliency method to skin lesion diagnosis to determine the effect of artefacts, such as ink, on model bias.