Deep multi-modal aggregation network for MR image reconstruction with auxiliary modality
This addresses the issue of long acquisition times and motion artifacts in MR imaging for medical applications, representing an incremental improvement by incorporating multi-modal data.
The paper tackles the problem of accelerating MR imaging by reconstructing full-sampled images from partial measurements, proposing a multi-modal aggregation network that uses an auxiliary modality to guide reconstruction and achieves superior artifact removal compared to state-of-the-art methods on IXI and fastMRI datasets.
Magnetic resonance (MR) imaging produces detailed images of organs and tissues with better contrast, but it suffers from a long acquisition time, which makes the image quality vulnerable to say motion artifacts. Recently, many approaches have been developed to reconstruct full-sampled images from partially observed measurements to accelerate MR imaging. However, most approaches focused on reconstruction over a single modality, neglecting the discovery of correlation knowledge between the different modalities. Here we propose a Multi-modal Aggregation network for mR Image recOnstruction with auxiliary modality (MARIO), which is capable of discovering complementary representations from a fully sampled auxiliary modality, with which to hierarchically guide the reconstruction of a given target modality. This implies that our method can selectively aggregate multi-modal representations for better reconstruction, yielding comprehensive, multi-scale, multi-modal feature fusion. Extensive experiments on IXI and fastMRI datasets demonstrate the superiority of the proposed approach over state-of-the-art MR image reconstruction methods in removing artifacts.