MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models
This work addresses the challenge of data scarcity in medical imaging for clinicians and researchers, though it is incremental as it applies an existing method to a new domain.
The authors tackled the problem of limited labeled data for chest X-ray pathology detection by adapting contrastive learning (MoCo) to medical imaging, resulting in improved model performance, such as outperforming non-pretrained models in detecting pleural effusion and showing benefits with limited labeled data.
Contrastive learning is a form of self-supervision that can leverage unlabeled data to produce pretrained models. While contrastive learning has demonstrated promising results on natural image classification tasks, its application to medical imaging tasks like chest X-ray interpretation has been limited. In this work, we propose MoCo-CXR, which is an adaptation of the contrastive learning method Momentum Contrast (MoCo), to produce models with better representations and initializations for the detection of pathologies in chest X-rays. In detecting pleural effusion, we find that linear models trained on MoCo-CXR-pretrained representations outperform those without MoCo-CXR-pretrained representations, indicating that MoCo-CXR-pretrained representations are of higher-quality. End-to-end fine-tuning experiments reveal that a model initialized via MoCo-CXR-pretraining outperforms its non-MoCo-CXR-pretrained counterpart. We find that MoCo-CXR-pretraining provides the most benefit with limited labeled training data. Finally, we demonstrate similar results on a target Tuberculosis dataset unseen during pretraining, indicating that MoCo-CXR-pretraining endows models with representations and transferability that can be applied across chest X-ray datasets and tasks.