Ariadne's Thread:Using Text Prompts to Improve Segmentation of Infected Areas from Chest X-ray images
This addresses the challenge of inaccurate segmentation in medical imaging for pulmonary infections, reducing the need for large annotated datasets.
The authors tackled the problem of segmenting infected lung areas in chest X-rays by proposing a language-driven method that uses text prompts, which improved the Dice score by at least 6.09% compared to uni-modal methods.
Segmentation of the infected areas of the lung is essential for quantifying the severity of lung disease like pulmonary infections. Existing medical image segmentation methods are almost uni-modal methods based on image. However, these image-only methods tend to produce inaccurate results unless trained with large amounts of annotated data. To overcome this challenge, we propose a language-driven segmentation method that uses text prompt to improve to the segmentation result. Experiments on the QaTa-COV19 dataset indicate that our method improves the Dice score by 6.09% at least compared to the uni-modal methods. Besides, our extended study reveals the flexibility of multi-modal methods in terms of the information granularity of text and demonstrates that multi-modal methods have a significant advantage over image-only methods in terms of the size of training data required.