Learnable Weight Initialization for Volumetric Medical Image Segmentation
This work addresses a bottleneck in medical image segmentation for researchers and practitioners by improving model performance without requiring external data, though it is incremental as it builds on existing hybrid architectures.
The paper tackles the problem of conventional weight initialization limiting hybrid volumetric medical image segmentation models by proposing a learnable, data-dependent initialization approach that leverages self-supervised objectives, achieving state-of-the-art performance on multi-organ and lung cancer segmentation tasks.
Hybrid volumetric medical image segmentation models, combining the advantages of local convolution and global attention, have recently received considerable attention. While mainly focusing on architectural modifications, most existing hybrid approaches still use conventional data-independent weight initialization schemes which restrict their performance due to ignoring the inherent volumetric nature of the medical data. To address this issue, we propose a learnable weight initialization approach that utilizes the available medical training data to effectively learn the contextual and structural cues via the proposed self-supervised objectives. Our approach is easy to integrate into any hybrid model and requires no external training data. Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach, leading to state-of-the-art segmentation performance. Our proposed data-dependent initialization approach performs favorably as compared to the Swin-UNETR model pretrained using large-scale datasets on multi-organ segmentation task. Our source code and models are available at: https://github.com/ShahinaKK/LWI-VMS.