SE3D: A Framework For Saliency Method Evaluation In 3D Imaging
It addresses the problem of evaluating explainability for 3D CNNs, which is crucial for safety in fields such as autonomous driving and medical imaging, but is incremental as it builds on existing 2D saliency evaluation concepts.
The paper tackles the lack of benchmarks for evaluating saliency methods in 3D imaging, proposing SE3D as a framework with modified datasets and metrics, and finds that current 3D saliency methods provide insufficient explanations, indicating room for improvement in critical applications like autonomous driving and medical imaging.
For more than a decade, deep learning models have been dominating in various 2D imaging tasks. Their application is now extending to 3D imaging, with 3D Convolutional Neural Networks (3D CNNs) being able to process LIDAR, MRI, and CT scans, with significant implications for fields such as autonomous driving and medical imaging. In these critical settings, explaining the model's decisions is fundamental. Despite recent advances in Explainable Artificial Intelligence, however, little effort has been devoted to explaining 3D CNNs, and many works explain these models via inadequate extensions of 2D saliency methods. A fundamental limitation to the development of 3D saliency methods is the lack of a benchmark to quantitatively assess these on 3D data. To address this issue, we propose SE3D: a framework for Saliency method Evaluation in 3D imaging. We propose modifications to ShapeNet, ScanNet, and BraTS datasets, and evaluation metrics to assess saliency methods for 3D CNNs. We evaluate both state-of-the-art saliency methods designed for 3D data and extensions of popular 2D saliency methods to 3D. Our experiments show that 3D saliency methods do not provide explanations of sufficient quality, and that there is margin for future improvements and safer applications of 3D CNNs in critical fields.