Rethinking Foundation Models for Medical Image Classification through a Benchmark Study on MedMNIST
This work addresses model selection for medical image classification, providing insights for researchers and practitioners, but it is incremental as it benchmarks existing models without introducing new methods.
The authors benchmarked various foundation models on the MedMNIST dataset for medical image classification, finding that pre-trained models show significant potential when transferred to these tasks, with results analyzed across different image sizes and training data amounts.
Foundation models are widely employed in medical image analysis, due to their high adaptability and generalizability for downstream tasks. With the increasing number of foundation models being released, model selection has become an important issue. In this work, we study the capabilities of foundation models in medical image classification tasks by conducting a benchmark study on the MedMNIST dataset. Specifically, we adopt various foundation models ranging from convolutional to Transformer-based models and implement both end-to-end training and linear probing for all classification tasks. The results demonstrate the significant potential of these pre-trained models when transferred for medical image classification. We further conduct experiments with different image sizes and various sizes of training data. By analyzing all the results, we provide preliminary, yet useful insights and conclusions on this topic.