Self-supervised vision-langage alignment of deep learning representations for bone X-rays analysis
This work addresses the challenge of medical image analysis for healthcare applications by integrating French reports, though it is incremental as it applies existing vision-language methods to a new language-specific domain.
The paper tackles the problem of analyzing bone X-rays by using self-supervised vision-language pretraining with French medical reports, achieving competitive performance on tasks like osteoarthritis quantification and fracture detection while reducing the need for expert annotations.
This paper proposes leveraging vision-language pretraining on bone X-rays paired with French reports to address downstream tasks of interest on bone radiography. A practical processing pipeline is introduced to anonymize and process French medical reports. Pretraining then consists in the self-supervised alignment of visual and textual embedding spaces derived from deep model encoders. The resulting image encoder is then used to handle various downstream tasks, including quantification of osteoarthritis, estimation of bone age on pediatric wrists, bone fracture and anomaly detection. Our approach demonstrates competitive performance on downstream tasks, compared to alternatives requiring a significantly larger amount of human expert annotations. Our work stands as the first study to integrate French reports to shape the embedding space devoted to bone X-Rays representations, capitalizing on the large quantity of paired images and reports data available in an hospital. By relying on generic vision-laguage deep models in a language-specific scenario, it contributes to the deployement of vision models for wider healthcare applications.