On the explainability of max-plus neural networks
Provides a new explainability method for a specific class of neural networks, with empirical validation on a medical imaging dataset.
The paper investigates explainability of max-plus neural networks, leveraging their property that a single most activated neuron determines output. They propose a pixel fragility measure and show it compares favorably to SHAP and Integrated Gradient on PneumoniaMnist.
We investigate the explanability properties of the recently proposed linear-min-max neural networks. At initialization, they can be interpreted as k-medoids with the infinity norm as a distance. Then, they are trained using subgradient descent to better fit the data. The model has been shown to be a universal approximator. Yet, we can trace the decision process because a single most activated neuron is responsible for the value of the output. Using this property, we designed a pixel fragility measure that determines whether changes to a single pixel may be responsible to a change in the classification output. Experiments on the PneumoniaMnist dataset show that this explanation for the output of the neural network compares favorably to SHAP and Integrated Gradient.