GLAM: Geometry-Guided Local Alignment for Multi-View VLP in Mammography
This work addresses the challenge of accurate and efficient mammography interpretation for breast cancer screening, though it is incremental as it builds on existing VLM methods with domain-specific adaptations.
The paper tackles the problem of developing a visual language model for mammography by addressing the lack of domain-specific multi-view relationships, proposing GLAM which uses geometry-guided alignment and contrastive learning to improve performance, achieving state-of-the-art results across multiple datasets.
Mammography screening is an essential tool for early detection of breast cancer. The speed and accuracy of mammography interpretation have the potential to be improved with deep learning methods. However, the development of a foundation visual language model (VLM) is hindered by limited data and domain differences between natural and medical images. Existing mammography VLMs, adapted from natural images, often ignore domain-specific characteristics, such as multi-view relationships in mammography. Unlike radiologists who analyze both views together to process ipsilateral correspondence, current methods treat them as independent images or do not properly model the multi-view correspondence learning, losing critical geometric context and resulting in suboptimal prediction. We propose GLAM: Global and Local Alignment for Multi-view mammography for VLM pretraining using geometry guidance. By leveraging the prior knowledge about the multi-view imaging process of mammograms, our model learns local cross-view alignments and fine-grained local features through joint global and local, visual-visual, and visual-language contrastive learning. Pretrained on EMBED [14], one of the largest open mammography datasets, our model outperforms baselines across multiple datasets under different settings.