Adversarial Synthesis Learning Enables Segmentation Without Target Modality Ground Truth
This work addresses the lack of generalizability in deep learning-based segmentation for medical imaging, enabling reuse of labels across modalities without manual effort, though it is incremental as it builds on prior two-stage methods.
The paper tackled the problem of segmenting organs in new imaging modalities without manual labels by proposing an end-to-end network that simultaneously synthesizes and segments images, achieving a median Dice similarity coefficient of 0.9188, outperforming existing methods.
A lack of generalizability is one key limitation of deep learning based segmentation. Typically, one manually labels new training images when segmenting organs in different imaging modalities or segmenting abnormal organs from distinct disease cohorts. The manual efforts can be alleviated if one is able to reuse manual labels from one modality (e.g., MRI) to train a segmentation network for a new modality (e.g., CT). Previously, two stage methods have been proposed to use cycle generative adversarial networks (CycleGAN) to synthesize training images for a target modality. Then, these efforts trained a segmentation network independently using synthetic images. However, these two independent stages did not use the complementary information between synthesis and segmentation. Herein, we proposed a novel end-to-end synthesis and segmentation network (EssNet) to achieve the unpaired MRI to CT image synthesis and CT splenomegaly segmentation simultaneously without using manual labels on CT. The end-to-end EssNet achieved significantly higher median Dice similarity coefficient (0.9188) than the two stages strategy (0.8801), and even higher than canonical multi-atlas segmentation (0.9125) and ResNet method (0.9107), which used the CT manual labels.