Competitive Multi-scale Convolution
This work addresses the training of complex deep learning models by reducing filter co-adaptation and dimensionality, offering incremental improvements for image classification tasks.
The paper tackles the problem of filter co-adaptation in deep convolutional neural networks by introducing a competitive multi-scale convolution module that replaces collaborative pooling with maxout activation, resulting in classification performance that is better than or comparable to state-of-the-art on benchmark datasets like MNIST, CIFAR-10, CIFAR-100, and SVHN.
In this paper, we introduce a new deep convolutional neural network (ConvNet) module that promotes competition among a set of multi-scale convolutional filters. This new module is inspired by the inception module, where we replace the original collaborative pooling stage (consisting of a concatenation of the multi-scale filter outputs) by a competitive pooling represented by a maxout activation unit. This extension has the following two objectives: 1) the selection of the maximum response among the multi-scale filters prevents filter co-adaptation and allows the formation of multiple sub-networks within the same model, which has been shown to facilitate the training of complex learning problems; and 2) the maxout unit reduces the dimensionality of the outputs from the multi-scale filters. We show that the use of our proposed module in typical deep ConvNets produces classification results that are either better than or comparable to the state of the art on the following benchmark datasets: MNIST, CIFAR-10, CIFAR-100 and SVHN.