On the effectiveness of GAN generated cardiac MRIs for segmentation
This work addresses the scarcity of labeled medical imaging data for researchers and clinicians, though it is incremental as it builds on existing GAN and VAE techniques.
The authors tackled the problem of limited annotated cardiac MRI data for segmentation by developing a VAE-GAN model that generates realistic synthetic MRIs with accurate ground truth, achieving up to a 12% improvement in Dice score when used for data augmentation.
In this work, we propose a Variational Autoencoder (VAE) - Generative Adversarial Networks (GAN) model that can produce highly realistic MRI together with its pixel accurate groundtruth for the application of cine-MR image cardiac segmentation. On one side of our model is a Variational Autoencoder (VAE) trained to learn the latent representations of cardiac shapes. On the other side is a GAN that uses "SPatially-Adaptive (DE)Normalization" (SPADE) modules to generate realistic MR images tailored to a given anatomical map. At test time, the sampling of the VAE latent space allows to generate an arbitrary large number of cardiac shapes, which are fed to the GAN that subsequently generates MR images whose cardiac structure fits that of the cardiac shapes. In other words, our system can generate a large volume of realistic yet labeled cardiac MR images. We show that segmentation with CNNs trained with our synthetic annotated images gets competitive results compared to traditional techniques. We also show that combining data augmentation with our GAN-generated images lead to an improvement in the Dice score of up to 12 percent while allowing for better generalization capabilities on other datasets.