Adaptive Convolution Kernel for Artificial Neural Networks
This addresses the problem of rigid kernel design in convolutional neural networks for researchers and practitioners in computer vision, offering an incremental improvement over standard methods.
The paper tackles the limitation of fixed-size convolutional kernels in deep neural networks by introducing a method to train adaptive kernel sizes within a single layer, using a differentiable Gaussian envelope. Results show statistically significant improvements in image classification across multiple datasets and enhanced learning performance and generalization in segmentation tasks.
Many deep neural networks are built by using stacked convolutional layers of fixed and single size (often 3$\times$3) kernels. This paper describes a method for training the size of convolutional kernels to provide varying size kernels in a single layer. The method utilizes a differentiable, and therefore backpropagation-trainable Gaussian envelope which can grow or shrink in a base grid. Our experiments compared the proposed adaptive layers to ordinary convolution layers in a simple two-layer network, a deeper residual network, and a U-Net architecture. The results in the popular image classification datasets such as MNIST, MNIST-CLUTTERED, CIFAR-10, Fashion, and ``Faces in the Wild'' showed that the adaptive kernels can provide statistically significant improvements on ordinary convolution kernels. A segmentation experiment in the Oxford-Pets dataset demonstrated that replacing a single ordinary convolution layer in a U-shaped network with a single 7$\times$7 adaptive layer can improve its learning performance and ability to generalize.