Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification
For medical imaging practitioners, this work provides a method to adapt ViTs to cardiac MR domain with improved performance, though the gains are incremental.
The paper addresses the poor transfer of pretrained Vision Transformer features to cardiac MR images by introducing a self-supervised contrastive learning adaptation strategy, achieving classification AUC > 0.75 across four common cardiac MR sequences.
Vision Transformer (ViT) models, utilizing self-attention mechanisms, have demonstrated robust generalization capabilities across various vision tasks, including image classification. However, these models, typically pretrained on general public datasets, often lack the specialized domain knowledge necessary for medical imaging applications. In this study, we investigate the adaptation of ViT models, specifically for cardiac magnetic resonance (MR) images, using an in-house dataset. We found that pretrained ViT features do not effectively transfer to the cardiac MR domain. To overcome this limitation, we introduce an adaptation strategy that utilizes image-based self-supervised contrastive learning, demonstrating superior performance compared to traditional supervised training approaches. Moreover, our adapted ViT model exhibits strong generalization to external MR datasets such as BraTS and ADNI. Through ablation studies, we further investigate the impact of batch size and dataset scale on performance. Ultimately, our adapted model achieves classification AUC exceeding 0.75 across the four most common cardiac MR sequences.