Efficient Multimodal Clinical Question Answering for Pulmonary Embolism Risk Assessment
For clinicians managing PE, this work provides a benchmark and initial evidence that compact multimodal models can leverage both imaging and EHR data for risk assessment, though results are preliminary and task-dependent.
This paper builds a benchmark for multimodal clinical question answering on pulmonary embolism (PE) risk assessment using 23,248 CTPA studies from 19,402 patients. Results show that compact multimodal models (Gemma4 E4B/E2B) perform best when combining CTPA and EHR data, with PE diagnosis outperforming prognostic tasks like readmission prediction.
Pulmonary embolism (PE) is a high risk cardiopulmonary condition whose management requires both timely diagnosis and reliable assessment of future clinical risk. Because PE care routinely combines computed tomography pulmonary angiography (CTPA), radiology interpretation, and longitudinal electronic health record (EHR) evidence, it provides a clinically meaningful setting for evaluating compact multimodal language models. In this work, we build a benchmark using efficient multimodal large language models (MLLMs) on INSPECT, a multimodal PE dataset containing 23,248 CTPA studies from 19,402 patients. We formulate eight diagnostic and prognostic tasks as structured clinical question answering problems and evaluate on typical efficient MLLMs under CTPA-Only, EHR-Only, and CTPA+EHR settings with zero-shot and few-shot prompting. Results show that Gemma4 E4B and Gemma4 E2B perform more strongly when EHR evidence is available, especially under CTPA+EHR input. Task level analysis further shows that PE diagnosis achieves higher performance than prognostic tasks, particularly readmission prediction. These observations suggest that compact multimodal models have the great potential in early stage PE risk detection and explanation.