MFPP: Morphological Fragmental Perturbation Pyramid for Black-Box Model Explanations
This addresses the need for explainable AI in applications like medical diagnosis and autonomous driving by providing a black-box interpretation method without requiring internal model access.
The paper tackles the problem of explaining black-box deep neural network predictions by proposing the Morphological Fragmental Perturbation Pyramid (MFPP) method, which divides input images into multi-scale fragments and uses random masking to generate saliency maps, and it demonstrates that MFPP meets and exceeds state-of-the-art performance on multiple models and datasets.
Deep neural networks (DNNs) have recently been applied and used in many advanced and diverse tasks, such as medical diagnosis, automatic driving, etc. Due to the lack of transparency of the deep models, DNNs are often criticized for their prediction that cannot be explainable by human. In this paper, we propose a novel Morphological Fragmental Perturbation Pyramid (MFPP) method to solve the Explainable AI problem. In particular, we focus on the black-box scheme, which can identify the input area that is responsible for the output of the DNN without having to understand the internal architecture of the DNN. In the MFPP method, we divide the input image into multi-scale fragments and randomly mask out fragments as perturbation to generate a saliency map, which indicates the significance of each pixel for the prediction result of the black box model. Compared with the existing input sampling perturbation method, the pyramid structure fragment has proved to be more effective. It can better explore the morphological information of the input image to match its semantic information, and does not need any value inside the DNN. We qualitatively and quantitatively prove that MFPP meets and exceeds the performance of state-of-the-art (SOTA) black-box interpretation method on multiple DNN models and datasets.