Memory-Efficient 3D High-Resolution Medical Image Synthesis Using CRF-Guided GANs
This work addresses memory constraints in 3D medical image synthesis, which is crucial for applications like diagnostic imaging and research, though it appears incremental as it builds on existing GAN and CRF techniques.
The paper tackles the problem of generating high-resolution 3D medical images with limited GPU memory by proposing a GAN architecture that uses Conditional Random Fields to model dependencies, enabling consistent image synthesis without memory exploitation. Experiments on Lung CTs and Brain MRIs show it outperforms state-of-the-art methods with lower memory usage and complexity.
Generative Adversarial Networks (GANs) have many potential medical imaging applications. Due to the limited memory of Graphical Processing Units (GPUs), most current 3D GAN models are trained on low-resolution medical images, these models cannot scale to high-resolution or are susceptible to patchy artifacts. In this work, we propose an end-to-end novel GAN architecture that uses Conditional Random field (CRF) to model dependencies so that it can generate consistent 3D medical Images without exploiting memory. To achieve this purpose, the generator is divided into two parts during training, the first part produces an intermediate representation and CRF is applied to this intermediate representation to capture correlations. The second part of the generator produces a random sub-volume of image using a subset of the intermediate representation. This structure has two advantages: first, the correlations are modeled by using the features that the generator is trying to optimize. Second, the generator can generate full high-resolution images during inference. Experiments on Lung CTs and Brain MRIs show that our architecture outperforms state-of-the-art while it has lower memory usage and less complexity.