Learning Cross-Modal Deep Representations for Multi-Modal MR Image Segmentation
This work addresses the challenge of efficient and accurate multi-modal MR image segmentation for clinical diagnosis and surgical planning, but it is incremental as it builds on existing CNN-based approaches.
The paper tackled the problem of suboptimal feature fusion in multi-modal MRI segmentation by proposing a supervised image fusion method with an attention block to select useful information and suppress noise, achieving better segmentation results than state-of-the-art methods on breast mass segmentation with two modalities.
Multi-modal magnetic resonance imaging (MRI) is essential in clinics for comprehensive diagnosis and surgical planning. Nevertheless, the segmentation of multi-modal MR images tends to be time-consuming and challenging. Convolutional neural network (CNN)-based multi-modal MR image analysis commonly proceeds with multiple down-sampling streams fused at one or several layers. Although inspiring performance has been achieved, the feature fusion is usually conducted through simple summation or concatenation without optimization. In this work, we propose a supervised image fusion method to selectively fuse the useful information from different modalities and suppress the respective noise signals. Specifically, an attention block is introduced as guidance for the information selection. From the different modalities, one modality that contributes most to the results is selected as the master modality, which supervises the information selection of the other assistant modalities. The effectiveness of the proposed method is confirmed through breast mass segmentation in MR images of two modalities and better segmentation results are achieved compared to the state-of-the-art methods.