3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models
This addresses the need for automated report generation in radiology, particularly for 3D medical images, but is incremental as it builds on prior VQA and multimodal models for a specific domain.
The paper tackled the problem of generating radiology reports from 3D CT scans, which is underexplored due to data and computational challenges, and introduced 3D-CT-GPT, a model that significantly outperforms existing methods in report accuracy and quality.
Medical image analysis is crucial in modern radiological diagnostics, especially given the exponential growth in medical imaging data. The demand for automated report generation systems has become increasingly urgent. While prior research has mainly focused on using machine learning and multimodal language models for 2D medical images, the generation of reports for 3D medical images has been less explored due to data scarcity and computational complexities. This paper introduces 3D-CT-GPT, a Visual Question Answering (VQA)-based medical visual language model specifically designed for generating radiology reports from 3D CT scans, particularly chest CTs. Extensive experiments on both public and private datasets demonstrate that 3D-CT-GPT significantly outperforms existing methods in terms of report accuracy and quality. Although current methods are few, including the partially open-source CT2Rep and the open-source M3D, we ensured fair comparison through appropriate data conversion and evaluation methodologies. Experimental results indicate that 3D-CT-GPT enhances diagnostic accuracy and report coherence, establishing itself as a robust solution for clinical radiology report generation. Future work will focus on expanding the dataset and further optimizing the model to enhance its performance and applicability.