Addressing Bias in VLMs for Glaucoma Detection Without Protected Attribute Supervision
This work addresses bias in automated glaucoma screening, a critical application for underserved populations, but it is incremental as it builds on existing contrastive learning frameworks.
The paper tackled bias in Vision-Language Models for glaucoma detection without using protected attributes, introducing an attribute-agnostic debiasing method that reduced subgroup disparities by upweighting underperforming clusters, as shown by metrics like Equalized Odds Distance and Equalized Subgroup AUC on the Harvard FairVLMed dataset.
Vision-Language Models (VLMs) have achieved remarkable success on multimodal tasks such as image-text retrieval and zero-shot classification, yet they can exhibit demographic biases even when explicit protected attributes are absent during training. In this work, we focus on automated glaucoma screening from retinal fundus images, a critical application given that glaucoma is a leading cause of irreversible blindness and disproportionately affects underserved populations. Building on a reweighting-based contrastive learning framework, we introduce an attribute-agnostic debiasing method that (i) infers proxy subgroups via unsupervised clustering of image-image embeddings, (ii) computes gradient-similarity weights between the CLIP-style multimodal loss and a SimCLR-style image-pair contrastive loss, and (iii) applies these weights in a joint, top-$k$ weighted objective to upweight underperforming clusters. This label-free approach adaptively targets the hardest examples, thereby reducing subgroup disparities. We evaluate our method on the Harvard FairVLMed glaucoma subset, reporting Equalized Odds Distance (EOD), Equalized Subgroup AUC (ES AUC), and Groupwise AUC to demonstrate equitable performance across inferred demographic subgroups.