Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization
This addresses domain generalization challenges in critical medical applications, offering a novel method to bridge causal gaps, though it is incremental in combining existing techniques.
The paper tackled the problem of generalizing image classification across domains in medical tasks like diabetic retinopathy grading and seizure detection, achieving superior single-source domain generalization with average accuracies of 69.2% and 81%, outperforming baselines by 9.4% and 1.8%.
Generalizing image classification across domains remains challenging in critical tasks such as fundus image-based diabetic retinopathy (DR) grading and resting-state fMRI seizure onset zone (SOZ) detection. When domains differ in unknown causal factors, achieving cross-domain generalization is difficult, and there is no established methodology to objectively assess such differences without direct metadata or protocol-level information from data collectors, which is typically inaccessible. We first introduce domain conformal bounds (DCB), a theoretical framework to evaluate whether domains diverge in unknown causal factors. Building on this, we propose GenEval, a multimodal Vision Language Models (VLM) approach that combines foundational models (e.g., MedGemma-4B) with human knowledge via Low-Rank Adaptation (LoRA) to bridge causal gaps and enhance single-source domain generalization (SDG). Across eight DR and two SOZ datasets, GenEval achieves superior SDG performance, with average accuracy of 69.2% (DR) and 81% (SOZ), outperforming the strongest baselines by 9.4% and 1.8%, respectively.