OCT Data is All You Need: How Vision Transformers with and without Pre-training Benefit Imaging
This work addresses the problem of optimizing pre-training strategies for medical imaging tasks like OCT classification, which is incremental as it builds on existing ViT methods.
The study investigated whether ImageNet pre-training benefits Vision Transformers for OCT image classification, finding that pre-training helps with small datasets but training from scratch can achieve comparable or superior accuracy with sufficient OCT data.
Optical Coherence Tomography (OCT) provides high-resolution cross-sectional images useful for diagnosing various diseases, but their distinct characteristics from natural images raise questions about whether large-scale pre-training on datasets like ImageNet is always beneficial. In this paper, we investigate the impact of ImageNet-based pre-training on Vision Transformer (ViT) performance for OCT image classification across different dataset sizes. Our experiments cover four-category retinal pathologies (CNV, DME, Drusen, Normal). Results suggest that while pre-training can accelerate convergence and potentially offer better performance in smaller datasets, training from scratch may achieve comparable or even superior accuracy when sufficient OCT data is available. Our findings highlight the importance of matching domain characteristics in pre-training and call for further study on large-scale OCT-specific pre-training.