Fusing Modalities by Multiplexed Graph Neural Networks for Outcome Prediction in Tuberculosis
This addresses the challenge of multimodal fusion for patient-tailored outcome prediction in tuberculosis, which is incremental as it builds on existing graph neural network approaches.
The paper tackled the problem of predicting tuberculosis outcomes by fusing multiple data modalities, and the proposed multiplexed graph neural network framework outperformed state-of-the-art methods on a large TB dataset.
In a complex disease such as tuberculosis, the evidence for the disease and its evolution may be present in multiple modalities such as clinical, genomic, or imaging data. Effective patient-tailored outcome prediction and therapeutic guidance will require fusing evidence from these modalities. Such multimodal fusion is difficult since the evidence for the disease may not be uniform across all modalities, not all modality features may be relevant, or not all modalities may be present for all patients. All these nuances make simple methods of early, late, or intermediate fusion of features inadequate for outcome prediction. In this paper, we present a novel fusion framework using multiplexed graphs and derive a new graph neural network for learning from such graphs. Specifically, the framework allows modalities to be represented through their targeted encodings, and models their relationship explicitly via multiplexed graphs derived from salient features in a combined latent space. We present results that show that our proposed method outperforms state-of-the-art methods of fusing modalities for multi-outcome prediction on a large Tuberculosis (TB) dataset.