CoMViT: An Efficient Vision Backbone for Supervised Classification in Medical Imaging
This work addresses efficiency and generalization challenges for resource-constrained medical image analysis, representing an incremental improvement with specific gains.
The paper tackled the problem of high computational demands and overfitting of Vision Transformers in medical imaging by introducing CoMViT, a compact architecture that achieves robust performance across twelve MedMNIST datasets with only ~4.5M parameters, matching or outperforming deeper models while offering up to 5-20x parameter reduction.
Vision Transformers (ViTs) have demonstrated strong potential in medical imaging; however, their high computational demands and tendency to overfit on small datasets limit their applicability in real-world clinical scenarios. In this paper, we present CoMViT, a compact and generalizable Vision Transformer architecture optimized for resource-constrained medical image analysis. CoMViT integrates a convolutional tokenizer, diagonal masking, dynamic temperature scaling, and pooling-based sequence aggregation to improve performance and generalization. Through systematic architectural optimization, CoMViT achieves robust performance across twelve MedMNIST datasets while maintaining a lightweight design with only ~4.5M parameters. It matches or outperforms deeper CNN and ViT variants, offering up to 5-20x parameter reduction without sacrificing accuracy. Qualitative Grad-CAM analyses show that CoMViT consistently attends to clinically relevant regions despite its compact size. These results highlight the potential of principled ViT redesign for developing efficient and interpretable models in low-resource medical imaging settings.