Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration
This addresses the problem of high computational costs for researchers and practitioners in medical imaging, enabling efficient large-scale 3D registration on low-VRAM GPUs, though it is incremental as it builds on existing U-Net architectures.
The paper tackles the computational inefficiency of U-Net style networks in unsupervised 3D medical image registration by proposing Fourier-Net and Fourier-Net+, which use band-limited Fourier representations to reduce resource use while achieving comparable results to state-of-the-art methods with faster inference, lower memory, and fewer operations.
U-Net style networks are commonly utilized in unsupervised image registration to predict dense displacement fields, which for high-resolution volumetric image data is a resource-intensive and time-consuming task. To tackle this challenge, we first propose Fourier-Net, which replaces the costly U-Net style expansive path with a parameter-free model-driven decoder. Instead of directly predicting a full-resolution displacement field, our Fourier-Net learns a low-dimensional representation of the displacement field in the band-limited Fourier domain which our model-driven decoder converts to a full-resolution displacement field in the spatial domain. Expanding upon Fourier-Net, we then introduce Fourier-Net+, which additionally takes the band-limited spatial representation of the images as input and further reduces the number of convolutional layers in the U-Net style network's contracting path. Finally, to enhance the registration performance, we propose a cascaded version of Fourier-Net+. We evaluate our proposed methods on three datasets, on which our proposed Fourier-Net and its variants achieve comparable results with current state-of-the art methods, while exhibiting faster inference speeds, lower memory footprint, and fewer multiply-add operations. With such small computational cost, our Fourier-Net+ enables the efficient training of large-scale 3D registration on low-VRAM GPUs. Our code is publicly available at \url{https://github.com/xi-jia/Fourier-Net}.