Application Of Vision-Language Models For Assessing Osteoarthritis Disease Severity
This addresses the need for precise and automated diagnostic methods for osteoarthritis, which is a global health challenge, by incorporating multimodal data to reduce variability in assessments.
This study tackled the problem of automating osteoarthritis severity assessment by using vision-language models to predict severity from X-ray images and corresponding reports, demonstrating efficacy in learning text-image representations and establishing a foundation for specialized models in medical contexts.
Osteoarthritis (OA) poses a global health challenge, demanding precise diagnostic methods. Current radiographic assessments are time consuming and prone to variability, prompting the need for automated solutions. The existing deep learning models for OA assessment are unimodal single task systems and they don't incorporate relevant text information such as patient demographics, disease history, or physician reports. This study investigates employing Vision Language Processing (VLP) models to predict OA severity using Xray images and corresponding reports. Our method leverages Xray images of the knee and diverse report templates generated from tabular OA scoring values to train a CLIP (Contrastive Language Image PreTraining) style VLP model. Furthermore, we incorporate additional contrasting captions to enforce the model to discriminate between positive and negative reports. Results demonstrate the efficacy of these models in learning text image representations and their contextual relationships, showcase potential advancement in OA assessment, and establish a foundation for specialized vision language models in medical contexts.