A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning
This work addresses the issue of hallucinations and inconsistent reasoning in medical diagnostic AI, which is critical for enhancing clinical trust, though it appears incremental as it builds upon existing models like LLaVA.
The paper tackles the problem of unreliable reasoning in multimodal medical AI by proposing a diagnostic framework that integrates vision-language models with logic tree reasoning, resulting in improved diagnostic accuracy and more interpretable reasoning traces on multimodal tasks.
With the rapid growth of large language models (LLMs) and vision-language models (VLMs) in medicine, simply integrating clinical text and medical imaging does not guarantee reliable reasoning. Existing multimodal models often produce hallucinations or inconsistent chains of thought, limiting clinical trust. We propose a diagnostic framework built upon LLaVA that combines vision-language alignment with logic-regularized reasoning. The system includes an input encoder for text and images, a projection module for cross-modal alignment, a reasoning controller that decomposes diagnostic tasks into steps, and a logic tree generator that assembles stepwise premises into verifiable conclusions. Evaluations on MedXpertQA and other benchmarks show that our method improves diagnostic accuracy and yields more interpretable reasoning traces on multimodal tasks, while remaining competitive on text-only settings. These results suggest a promising step toward trustworthy multimodal medical AI.