K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging
This work addresses disease diagnosis in medical imaging, offering a novel approach to handle long-tailed and zero-shot recognition, which is incremental in combining knowledge graphs with visual encoders.
The paper tackles disease diagnosis in radiographic imaging by proposing a knowledge-enhanced framework that incorporates medical domain knowledge to improve visual representation learning, achieving benefits for long-tailed and zero-shot recognition problems where conventional methods struggle.
In this paper, we consider the problem of disease diagnosis. Unlike the conventional learning paradigm that treats labels independently, we propose a knowledge-enhanced framework, that enables training visual representation with the guidance of medical domain knowledge. In particular, we make the following contributions: First, to explicitly incorporate experts' knowledge, we propose to learn a neural representation for the medical knowledge graph via contrastive learning, implicitly establishing relations between different medical concepts. Second, while training the visual encoder, we keep the parameters of the knowledge encoder frozen and propose to learn a set of prompt vectors for efficient adaptation. Third, we adopt a Transformer-based disease-query module for cross-model fusion, which naturally enables explainable diagnosis results via cross attention. To validate the effectiveness of our proposed framework, we conduct thorough experiments on three x-ray imaging datasets across different anatomy structures, showing our model is able to exploit the implicit relations between diseases/findings, thus is beneficial to the commonly encountered problem in the medical domain, namely, long-tailed and zero-shot recognition, which conventional methods either struggle or completely fail to realize.