CNN-based explanation ensembling for dataset, representation and explanations evaluation
This work addresses the need for more reliable and coherent explanations in high-stake domains like medicine and finance, though it appears incremental as it builds on existing explanation methods.
The paper tackles the problem of inconsistent explanations from deep learning models by proposing an ensembling method for CNN-based explanations, demonstrating superior performance in localization and faithfulness metrics compared to individual explanations.
Explainable Artificial Intelligence has gained significant attention due to the widespread use of complex deep learning models in high-stake domains such as medicine, finance, and autonomous cars. However, different explanations often present different aspects of the model's behavior. In this research manuscript, we explore the potential of ensembling explanations generated by deep classification models using convolutional model. Through experimentation and analysis, we aim to investigate the implications of combining explanations to uncover a more coherent and reliable patterns of the model's behavior, leading to the possibility of evaluating the representation learned by the model. With our method, we can uncover problems of under-representation of images in a certain class. Moreover, we discuss other side benefits like features' reduction by replacing the original image with its explanations resulting in the removal of some sensitive information. Through the use of carefully selected evaluation metrics from the Quantus library, we demonstrated the method's superior performance in terms of Localisation and Faithfulness, compared to individual explanations.