Hybrid-Fusion Transformer for Multisequence MRI
This work addresses the challenge of multimodal MRI segmentation for medical imaging applications, representing an incremental improvement by combining existing Transformer and fusion techniques.
The authors tackled the problem of integrating different MRI sequence characteristics into Transformer models for medical segmentation, proposing a hybrid fusion Transformer (HFTrans) that outperformed previous state-of-the-art methods on brain tumor and structure segmentation tasks using BraTS2020 and MRBrainS18 datasets.
Medical segmentation has grown exponentially through the advent of a fully convolutional network (FCN), and we have now reached a turning point through the success of Transformer. However, the different characteristics of the modality have not been fully integrated into Transformer for medical segmentation. In this work, we propose the novel hybrid fusion Transformer (HFTrans) for multisequence MRI image segmentation. We take advantage of the differences among multimodal MRI sequences and utilize the Transformer layers to integrate the features extracted from each modality as well as the features of the early fused modalities. We validate the effectiveness of our hybrid-fusion method in three-dimensional (3D) medical segmentation. Experiments on two public datasets, BraTS2020 and MRBrainS18, show that the proposed method outperforms previous state-of-the-art methods on the task of brain tumor segmentation and brain structure segmentation.