LoFi: Location-Aware Fine-Grained Representation Learning for Chest X-ray
This addresses the challenge of capturing spatially confined clinical findings in chest X-rays for medical imaging tasks, representing an incremental improvement over existing methods.
The paper tackled the problem of fine-grained representation learning for chest X-ray retrieval and phrase grounding by proposing LoFi, which uses location-aware captioning losses to provide region-level supervision, resulting in superior performance on MIMIC-CXR and PadChest-GR datasets.
Fine-grained representation learning is crucial for retrieval and phrase grounding in chest X-rays, where clinically relevant findings are often spatially confined. However, the lack of region-level supervision in contrastive models and the limited ability of large vision language models to capture fine-grained representations in external validation lead to suboptimal performance on these tasks. To address these limitations, we propose Location-aware Fine-grained representation learning (LoFi), which jointly optimizes sigmoid, captioning, and location-aware captioning losses using a lightweight large language model. The location-aware captioning loss enables region-level supervision through grounding and dense captioning objectives, thereby facilitating fine-grained representation learning. Building upon these representations, we integrate a fine-grained encoder into retrieval-based in-context learning to enhance chest X-ray grounding across diverse settings. Extensive experiments demonstrate that our method achieves superior retrieval and phrase grounding performance on MIMIC-CXR and PadChest-GR.