MMLN: Leveraging Domain Knowledge for Multimodal Diagnosis
This work addresses the challenge of interpretable and data-efficient diagnosis for medical professionals, though it is incremental in combining existing knowledge-driven and data-driven approaches.
The paper tackled the problem of multimodal lung disease diagnosis by integrating domain knowledge from clinical guidelines with data-driven learning, resulting in improved accuracy and interpretability over state-of-the-art baselines on a real-world hospital dataset.
Recent studies show that deep learning models achieve good performance on medical imaging tasks such as diagnosis prediction. Among the models, multimodality has been an emerging trend, integrating different forms of data such as chest X-ray (CXR) images and electronic medical records (EMRs). However, most existing methods incorporate them in a model-free manner, which lacks theoretical support and ignores the intrinsic relations between different data sources. To address this problem, we propose a knowledge-driven and data-driven framework for lung disease diagnosis. By incorporating domain knowledge, machine learning models can reduce the dependence on labeled data and improve interpretability. We formulate diagnosis rules according to authoritative clinical medicine guidelines and learn the weights of rules from text data. Finally, a multimodal fusion consisting of text and image data is designed to infer the marginal probability of lung disease. We conduct experiments on a real-world dataset collected from a hospital. The results show that the proposed method outperforms the state-of-the-art multimodal baselines in terms of accuracy and interpretability.