Can Prompt Learning Benefit Radiology Report Generation?
This addresses the challenge of generating clinically meaningful radiology reports, which is a domain-specific problem in medical AI, and is incremental as it applies prompt learning to a new task.
The paper tackles radiology report generation by proposing PromptRRG, a method using prompt learning to incorporate prior medical knowledge, achieving state-of-the-art performance on the MIMIC-CXR benchmark.
Radiology report generation aims to automatically provide clinically meaningful descriptions of radiology images such as MRI and X-ray. Although great success has been achieved in natural scene image captioning tasks, radiology report generation remains challenging and requires prior medical knowledge. In this paper, we propose PromptRRG, a method that utilizes prompt learning to activate a pretrained model and incorporate prior knowledge. Since prompt learning for radiology report generation has not been explored before, we begin with investigating prompt designs and categorise them based on varying levels of knowledge: common, domain-specific and disease-enriched prompts. Additionally, we propose an automatic prompt learning mechanism to alleviate the burden of manual prompt engineering. This is the first work to systematically examine the effectiveness of prompt learning for radiology report generation. Experimental results on the largest radiology report generation benchmark, MIMIC-CXR, demonstrate that our proposed method achieves state-of-the-art performance. Code will be available upon the acceptance.