Truly shift-equivariant convolutional neural networks with adaptive polyphase upsampling
This addresses the shift equivariance issue in CNNs for image reconstruction, benefiting medical imaging applications, and is incremental as it builds on prior APS-D work.
The paper tackled the problem of shift equivariance in convolutional neural networks for image reconstruction tasks by proposing adaptive polyphase upsampling (APS-U), which, combined with APS-D, enables perfect shift equivariance in symmetric encoder-decoder architectures like U-Net. Results from MRI and CT reconstruction experiments showed state-of-the-art equivariance performance without compromising image quality, with gains extending to out-of-distribution images.
Convolutional neural networks lack shift equivariance due to the presence of downsampling layers. In image classification, adaptive polyphase downsampling (APS-D) was recently proposed to make CNNs perfectly shift invariant. However, in networks used for image reconstruction tasks, it can not by itself restore shift equivariance. We address this problem by proposing adaptive polyphase upsampling (APS-U), a non-linear extension of conventional upsampling, which allows CNNs with symmetric encoder-decoder architecture (for example U-Net) to exhibit perfect shift equivariance. With MRI and CT reconstruction experiments, we show that networks containing APS-D/U layers exhibit state of the art equivariance performance without sacrificing on image reconstruction quality. In addition, unlike prior methods like data augmentation and anti-aliasing, the gains in equivariance obtained from APS-D/U also extend to images outside the training distribution.