Multi-Plane Vision Transformer for Hemorrhage Classification Using Axial and Sagittal MRI Data
This work addresses a domain-specific problem for healthcare professionals by improving hemorrhage detection in MRI with varying orientations, though it is incremental as it builds on existing vision transformer methods.
The paper tackled the problem of brain hemorrhage classification from MRI data with varying orientations by proposing a 3D multi-plane vision transformer (MP-ViT) that integrates axial and sagittal contrasts using cross-attention, achieving a 5.5% improvement in AUC over a vision transformer and 1.8% over CNN-based architectures on a clinical dataset.
Identifying brain hemorrhages from magnetic resonance imaging (MRI) is a critical task for healthcare professionals. The diverse nature of MRI acquisitions with varying contrasts and orientation introduce complexity in identifying hemorrhage using neural networks. For acquisitions with varying orientations, traditional methods often involve resampling images to a fixed plane, which can lead to information loss. To address this, we propose a 3D multi-plane vision transformer (MP-ViT) for hemorrhage classification with varying orientation data. It employs two separate transformer encoders for axial and sagittal contrasts, using cross-attention to integrate information across orientations. MP-ViT also includes a modality indication vector to provide missing contrast information to the model. The effectiveness of the proposed model is demonstrated with extensive experiments on real world clinical dataset consists of 10,084 training, 1,289 validation and 1,496 test subjects. MP-ViT achieved substantial improvement in area under the curve (AUC), outperforming the vision transformer (ViT) by 5.5% and CNN-based architectures by 1.8%. These results highlight the potential of MP-ViT in improving performance for hemorrhage detection when different orientation contrasts are needed.