FNOSeg3D: Resolution-Robust 3D Image Segmentation with Fourier Neural Operator
This addresses the computational and accuracy challenges in 3D medical image segmentation for healthcare applications, representing an incremental improvement by adapting the Fourier neural operator.
The paper tackled the problem of 3D medical image segmentation accuracy dropping when models trained on downsampled images are applied to original resolutions, and introduced FNOSeg3D, which achieved superior robustness to training image resolution with less than 1% of the parameters of other models on the BraTS'19 dataset.
Due to the computational complexity of 3D medical image segmentation, training with downsampled images is a common remedy for out-of-memory errors in deep learning. Nevertheless, as standard spatial convolution is sensitive to variations in image resolution, the accuracy of a convolutional neural network trained with downsampled images can be suboptimal when applied on the original resolution. To address this limitation, we introduce FNOSeg3D, a 3D segmentation model robust to training image resolution based on the Fourier neural operator (FNO). The FNO is a deep learning framework for learning mappings between functions in partial differential equations, which has the appealing properties of zero-shot super-resolution and global receptive field. We improve the FNO by reducing its parameter requirement and enhancing its learning capability through residual connections and deep supervision, and these result in our FNOSeg3D model which is parameter efficient and resolution robust. When tested on the BraTS'19 dataset, it achieved superior robustness to training image resolution than other tested models with less than 1% of their model parameters.