Dual Adversarial Learning with Attention Mechanism for Fine-grained Medical Image Synthesis
This work addresses a domain-specific problem in medical imaging by improving synthesis accuracy for critical regions like tumors, which is incremental as it builds on existing cross-modality synthesis methods.
The paper tackles the problem of synthesizing fine-grained medical images, such as generating T2 MRI from T1 MRI for brain tumors or MRI from CT, by addressing hard-to-synthesize regions like tumors. It proposes a dual-discriminator adversarial learning system with an attention mechanism, outperforming state-of-the-art methods on all evaluated datasets and tasks.
Medical imaging plays a critical role in various clinical applications. However, due to multiple considerations such as cost and risk, the acquisition of certain image modalities could be limited. To address this issue, many cross-modality medical image synthesis methods have been proposed. However, the current methods cannot well model the hard-to-synthesis regions (e.g., tumor or lesion regions). To address this issue, we propose a simple but effective strategy, that is, we propose a dual-discriminator (dual-D) adversarial learning system, in which, a global-D is used to make an overall evaluation for the synthetic image, and a local-D is proposed to densely evaluate the local regions of the synthetic image. More importantly, we build an adversarial attention mechanism which targets at better modeling hard-to-synthesize regions (e.g., tumor or lesion regions) based on the local-D. Experimental results show the robustness and accuracy of our method in synthesizing fine-grained target images from the corresponding source images. In particular, we evaluate our method on two datasets, i.e., to address the tasks of generating T2 MRI from T1 MRI for the brain tumor images and generating MRI from CT. Our method outperforms the state-of-the-art methods under comparison in all datasets and tasks. And the proposed difficult-region-aware attention mechanism is also proved to be able to help generate more realistic images, especially for the hard-to-synthesize regions.