Ordinal Pooling Networks: For Preserving Information over Shrinking Feature Maps
This addresses the issue of information loss during downsampling in deep learning for image classification, though it appears incremental as it modifies an existing pooling method rather than introducing a new paradigm.
The paper tackles the problem of information loss in max-pooling operations in convolutional neural networks by introducing Ordinal Pooling Networks (OPN), which assign learned weights to all elements in a pooling region based on their order, resulting in consistent accuracy improvements over max-pooling in small-scale image classification experiments.
In the framework of convolutional neural networks that lie at the heart of deep learning, downsampling is often performed with a max-pooling operation that only retains the element with maximum activation, while completely discarding the information contained in other elements in a pooling region. To address this issue, a novel pooling scheme, Ordinal Pooling Network (OPN), is introduced in this work. OPN rearranges all the elements of a pooling region in a sequence and assigns different weights to these elements based upon their orders in the sequence, where the weights are learned via the gradient-based optimisation. The results of our small-scale experiments on image classification task demonstrate that this scheme leads to a consistent improvement in the accuracy over max-pooling operation. This improvement is expected to increase in deeper networks, where several layers of pooling become necessary.