Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks
This work addresses the problem of shift inconsistency in convolutional networks for tasks like image classification and semantic segmentation, offering a novel solution that improves performance and consistency, though it is incremental as it builds on existing downsampling methods.
The paper tackles the problem of achieving shift invariance and equivariance in convolutional networks by proposing learnable polyphase sampling (LPS), a pair of learnable down/upsampling layers that can be trained end-to-end and integrated into any convolutional network. The result shows that LPS achieves true shift-equivariance on semantic segmentation with 100% shift consistency, outperforming baselines by an absolute 3.3% on PASCAL VOC.
We propose learnable polyphase sampling (LPS), a pair of learnable down/upsampling layers that enable truly shift-invariant and equivariant convolutional networks. LPS can be trained end-to-end from data and generalizes existing handcrafted downsampling layers. It is widely applicable as it can be integrated into any convolutional network by replacing down/upsampling layers. We evaluate LPS on image classification and semantic segmentation. Experiments show that LPS is on-par with or outperforms existing methods in both performance and shift consistency. For the first time, we achieve true shift-equivariance on semantic segmentation (PASCAL VOC), i.e., 100% shift consistency, outperforming baselines by an absolute 3.3%.