When is a Foundation Model a Foundation Model
This work addresses the effectiveness of foundation models in specialized medical domains, highlighting potential limitations for practitioners in digital pathology.
The study found that fine-tuned foundation models for medical image-text tasks, trained on online data like Twitter and PubMed, perform worse in retrieval tasks for digital pathology compared to smaller traditional deep networks, with inferior performance observed in validation.
Recently, several studies have reported on the fine-tuning of foundation models for image-text modeling in the field of medicine, utilizing images from online data sources such as Twitter and PubMed. Foundation models are large, deep artificial neural networks capable of learning the context of a specific domain through training on exceptionally extensive datasets. Through validation, we have observed that the representations generated by such models exhibit inferior performance in retrieval tasks within digital pathology when compared to those generated by significantly smaller, conventional deep networks.