AMANDA: Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering
This work addresses the bottleneck in medical reasoning for Med-MLLMs in data-scarce environments, offering a domain-specific solution for medical AI applications.
The paper tackles the problem of medical visual question answering in low-resource settings by proposing AMANDA, a training-free agentic framework that augments medical knowledge through LLM agents, resulting in substantial improvements across eight benchmarks in zero-shot and few-shot settings.
Medical Multimodal Large Language Models (Med-MLLMs) have shown great promise in medical visual question answering (Med-VQA). However, when deployed in low-resource settings where abundant labeled data are unavailable, existing Med-MLLMs commonly fail due to their medical reasoning capability bottlenecks: (i) the intrinsic reasoning bottleneck that ignores the details from the medical image; (ii) the extrinsic reasoning bottleneck that fails to incorporate specialized medical knowledge. To address those limitations, we propose AMANDA, a training-free agentic framework that performs medical knowledge augmentation via LLM agents. Specifically, our intrinsic medical knowledge augmentation focuses on coarse-to-fine question decomposition for comprehensive diagnosis, while extrinsic medical knowledge augmentation grounds the reasoning process via biomedical knowledge graph retrieval. Extensive experiments across eight Med-VQA benchmarks demonstrate substantial improvements in both zero-shot and few-shot Med-VQA settings. The code is available at https://github.com/REAL-Lab-NU/AMANDA.