Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks
This work addresses a validation gap in deep learning for medical imaging by showing that specialized pretraining can enhance performance on small pathological benchmarks, though it is incremental as it builds on existing pretraining methods.
The study tackled the problem of whether pretraining on a large specialized dataset improves fine-tuning on small similar datasets in pathology imaging, finding that self-supervised pretraining on a large hematoxylin and eosin-stained image dataset (PTCGA200) outperformed general pretraining (e.g., ImageNet2012) in tumor classification (accuracy up to 86.41%) and tissue segmentation (mIoU up to 63.53%).
Pretraining a deep learning model on large image datasets is a standard step before fine-tuning the model on small targeted datasets. The large dataset is usually general images (e.g. imagenet2012) while the small dataset can be specialized datasets that have different distributions from the large dataset. However, this 'large-to-small' strategy is not well-validated when the large dataset is specialized and has a similar distribution to small datasets. We newly compiled three hematoxylin and eosin-stained image datasets, one large (PTCGA200) and two magnification-adjusted small datasets (PCam200 and segPANDA200). Major deep learning models were trained with supervised and self-supervised learning methods and fine-tuned on the small datasets for tumor classification and tissue segmentation benchmarks. ResNet50 pretrained with MoCov2, SimCLR, and BYOL on PTCGA200 was better than imagenet2012 pretraining when fine-tuned on PTCGA200 (accuracy of 83.94%, 86.41%, 84.91%, and 82.72%, respectively). ResNet50 pre-trained on PTCGA200 with MoCov2 exceeded the COCOtrain2017-pretrained baseline and was the best in ResNet50 for the tissue segmentation benchmark (mIoU of 63.53% and 63.22%). We found re-training imagenet-pretrained models (ResNet50, BiT-M-R50x1, and ViT-S/16) on PTCGA200 improved downstream benchmarks.