Weakly Supervised Concept Learning with Class-Level Priors for Interpretable Medical Diagnosis
This addresses the need for reliable, human-interpretable AI in medical imaging, particularly in clinical contexts where annotations are impractical, though it is incremental as it builds on existing interpretable-by-design frameworks.
The paper tackles the problem of enabling interpretable medical diagnosis without costly concept annotations by proposing a weakly supervised framework that improves concept-level F1-score by over 33% compared to zero-shot baselines while maintaining competitive classification performance.
Human-interpretable predictions are essential for deploying AI in medical imaging, yet most interpretable-by-design (IBD) frameworks require concept annotations for training data, which are costly and impractical to obtain in clinical contexts. Recent attempts to bypass annotation, such as zero-shot vision-language models or concept-generation frameworks, struggle to capture domain-specific medical features, leading to poor reliability. In this paper, we propose a novel Prior-guided Concept Predictor (PCP), a weakly supervised framework that enables concept answer prediction without explicit supervision or reliance on language models. PCP leverages class-level concept priors as weak supervision and incorporates a refinement mechanism with KL divergence and entropy regularization to align predictions with clinical reasoning. Experiments on PH2 (dermoscopy) and WBCatt (hematology) show that PCP improves concept-level F1-score by over 33% compared to zero-shot baselines, while delivering competitive classification performance on four medical datasets (PH2, WBCatt, HAM10000, and CXR4) relative to fully supervised concept bottleneck models (CBMs) and V-IP.