Fusing Medical Image Features and Clinical Features with Deep Learning for Computer-Aided Diagnosis
This addresses the need for more comprehensive CAD systems in medical practice by fusing multimodal data, though it is incremental as it builds on existing deep learning approaches.
The paper tackled the problem of integrating clinical information with medical images for computer-aided diagnosis, proposing a deep learning method that uses clinical features to guide image feature extraction and achieves improved diagnostic performance across tasks like Alzheimer's disease and hepatic microvascular invasion.
Current Computer-Aided Diagnosis (CAD) methods mainly depend on medical images. The clinical information, which usually needs to be considered in practical clinical diagnosis, has not been fully employed in CAD. In this paper, we propose a novel deep learning-based method for fusing Magnetic Resonance Imaging (MRI)/Computed Tomography (CT) images and clinical information for diagnostic tasks. Two paths of neural layers are performed to extract image features and clinical features, respectively, and at the same time clinical features are employed as the attention to guide the extraction of image features. Finally, these two modalities of features are concatenated to make decisions. We evaluate the proposed method on its applications to Alzheimer's disease diagnosis, mild cognitive impairment converter prediction and hepatic microvascular invasion diagnosis. The encouraging experimental results prove the values of the image feature extraction guided by clinical features and the concatenation of two modalities of features for classification, which improve the performance of diagnosis effectively and stably.