Explainable AI Approach using Near Misses Analysis
This addresses the need for explainable AI in neural networks, offering a novel method that could impact XAI research, though it appears incremental in its application to existing models.
The paper tackles the problem of explaining neural network decisions by introducing a near-misses analysis approach that infers logical concepts from latent processes without examining network structure, tested on architectures like ResNet and datasets like ImageNet, showing it can reflect concept generation and revealing that efficient architectures may sacrifice explainability and robustness despite similar accuracy.
This paper introduces a novel XAI approach based on near-misses analysis (NMA). This approach reveals a hierarchy of logical 'concepts' inferred from the latent decision-making process of a Neural Network (NN) without delving into its explicit structure. We examined our proposed XAI approach on different network architectures that vary in size and shape (e.g., ResNet, VGG, EfficientNet, MobileNet) on several datasets (ImageNet and CIFAR100). The results demonstrate its usability to reflect NNs latent process of concepts generation. We generated a new metric for explainability. Moreover, our experiments suggest that efficient architectures, which achieve a similar accuracy level with much less neurons may still pay the price of explainability and robustness in terms of concepts generation. We, thus, pave a promising new path for XAI research to follow.