Towards a Visual-Language Foundation Model for Computational Pathology
This work addresses the challenge of building versatile AI models for pathology that can handle multiple tasks with minimal fine-tuning, potentially benefiting medical diagnostics and research.
The authors tackled the problem of label scarcity and task-specific limitations in computational pathology by developing CONCH, a visual-language foundation model pretrained on over 1.17 million image-caption pairs, which achieved state-of-the-art performance on 13 diverse benchmarks including classification, segmentation, and retrieval tasks.
The accelerated adoption of digital pathology and advances in deep learning have enabled the development of powerful models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain and the model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text, and notably over 1.17 million image-caption pairs via task-agnostic pretraining. Evaluated on a suite of 13 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving either or both histopathology images and text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning.