Universal Pooling -- A New Pooling Method for Convolutional Neural Networks
This addresses a limitation in CNN design for researchers and practitioners, though it appears incremental as it builds on attention methods and existing pooling techniques.
The paper tackles the problem of fixed pooling functions in convolutional neural networks by proposing universal pooling, a method that generates any pooling function depending on the problem and dataset, and it outperformed existing pooling methods on two benchmark problems.
Pooling is one of the main elements in convolutional neural networks. The pooling reduces the size of the feature map, enabling training and testing with a limited amount of computation. This paper proposes a new pooling method named universal pooling. Unlike the existing pooling methods such as average pooling, max pooling, and stride pooling with fixed pooling function, universal pooling generates any pooling function, depending on a given problem and dataset. Universal pooling was inspired by attention methods and can be considered as a channel-wise form of local spatial attention. Universal pooling is trained jointly with the main network and it is shown that it includes the existing pooling methods. Finally, when applied to two benchmark problems, the proposed method outperformed the existing pooling methods and performed with the expected diversity, adapting to the given problem.