Bort: Towards Explainable Neural Networks with Bounded Orthogonal Constraint
This addresses the problem of black-box models in reliability-demanded industries, offering a novel method with mathematical guarantees, though it appears incremental in the context of existing explainability research.
The paper tackles the lack of mathematical guarantees in explainable neural networks by introducing Bort, an optimizer with bounded orthogonal constraints, which improves model explainability and classification accuracy, achieving gains on datasets like MNIST, CIFAR-10, and ImageNet.
Deep learning has revolutionized human society, yet the black-box nature of deep neural networks hinders further application to reliability-demanded industries. In the attempt to unpack them, many works observe or impact internal variables to improve the comprehensibility and invertibility of the black-box models. However, existing methods rely on intuitive assumptions and lack mathematical guarantees. To bridge this gap, we introduce Bort, an optimizer for improving model explainability with boundedness and orthogonality constraints on model parameters, derived from the sufficient conditions of model comprehensibility and invertibility. We perform reconstruction and backtracking on the model representations optimized by Bort and observe a clear improvement in model explainability. Based on Bort, we are able to synthesize explainable adversarial samples without additional parameters and training. Surprisingly, we find Bort constantly improves the classification accuracy of various architectures including ResNet and DeiT on MNIST, CIFAR-10, and ImageNet. Code: https://github.com/zbr17/Bort.