Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning
This work addresses the problem of generating high-quality medical reports for radiologists, but it is incremental as it builds on existing methods like SAM and contrastive learning.
The paper tackled automated radiology report generation by proposing a framework that uses segmentation and contrastive learning to improve visual representations and reduce data bias, achieving state-of-the-art performance on the IU X-Ray dataset.
Automated radiology report generation has the potential to improve radiology reporting and alleviate the workload of radiologists. However, the medical report generation task poses unique challenges due to the limited availability of medical data and the presence of data bias. To maximize the utility of available data and reduce data bias, we propose MSCL (Medical image Segmentation with Contrastive Learning), a framework that utilizes the Segment Anything Model (SAM) to segment organs, abnormalities, bones, etc., and can pay more attention to the meaningful ROIs in the image to get better visual representations. Then we introduce a supervised contrastive loss that assigns more weight to reports that are semantically similar to the target while training. The design of this loss function aims to mitigate the impact of data bias and encourage the model to capture the essential features of a medical image and generate high-quality reports. Experimental results demonstrate the effectiveness of our proposed model, where we achieve state-of-the-art performance on the IU X-Ray public dataset.