NullSpaceNet: Nullspace Convoluional Neural Network with Differentiable Loss Function
This addresses the need for more interpretable and efficient deep learning models for image classification tasks, though it appears incremental as it builds on existing CNN architectures.
The authors tackled the problem of improving interpretability and efficiency in convolutional neural networks by mapping inputs to a joint-nullspace instead of traditional feature spaces, resulting in up to 4.55% accuracy gain, a reduction from 135M to 19M parameters, and 99% faster inference time compared to VGG16.
We propose NullSpaceNet, a novel network that maps from the pixel level input to a joint-nullspace (as opposed to the traditional feature space), where the newly learned joint-nullspace features have clearer interpretation and are more separable. NullSpaceNet ensures that all inputs from the same class are collapsed into one point in this new joint-nullspace, and the different classes are collapsed into different points with high separation margins. Moreover, a novel differentiable loss function is proposed that has a closed-form solution with no free-parameters. NullSpaceNet exhibits superior performance when tested against VGG16 with fully-connected layer over 4 different datasets, with accuracy gain of up to 4.55%, a reduction in learnable parameters from 135M to 19M, and reduction in inference time of 99% in favor of NullSpaceNet. This means that NullSpaceNet needs less than 1% of the time it takes a traditional CNN to classify a batch of images with better accuracy.