Med-SORA: Symptom to Organ Reasoning in Abdomen CT Images
This addresses the need for more realistic clinical reasoning in medical AI by handling one-to-many symptom-organ relationships, though it is incremental in improving multimodal methods for a specific domain.
The paper tackled the problem of symptom-to-organ reasoning in abdominal CT images by proposing Med-SORA, which uses soft labeling and a 2D-3D cross-attention architecture, and it outperforms existing models in enabling accurate 3D clinical reasoning.
Understanding symptom-image associations is crucial for clinical reasoning. However, existing medical multimodal models often rely on simple one-to-one hard labeling, oversimplifying clinical reality where symptoms relate to multiple organs. In addition, they mainly use single-slice 2D features without incorporating 3D information, limiting their ability to capture full anatomical context. In this study, we propose Med-SORA, a framework for symptom-to-organ reasoning in abdominal CT images. Med-SORA introduces RAG-based dataset construction, soft labeling with learnable organ anchors to capture one-to-many symptom-organ relationships, and a 2D-3D cross-attention architecture to fuse local and global image features. To our knowledge, this is the first work to address symptom-to-organ reasoning in medical multimodal learning. Experimental results show that Med-SORA outperforms existing medical multimodal models and enables accurate 3D clinical reasoning.