Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images
This addresses the computational bottleneck for generating high-resolution medical images, which is crucial for medical imaging applications, though it is an incremental improvement over existing patch-based approaches.
The paper tackles the problem of generating high-resolution medical images with GANs, which is computationally demanding for large sizes like 3D volumes, by proposing a multi-scale patch-based GAN that learns low-resolution images first and then generates patches at higher resolutions, enabling generation of arbitrarily large images like 512x512x512 3D CTs and 2048x2048 X-rays with constant GPU memory and better quality than common patch-based methods.
Currently generative adversarial networks (GANs) are rarely applied to medical images of large sizes, especially 3D volumes, due to their large computational demand. We propose a novel multi-scale patch-based GAN approach to generate large high resolution 2D and 3D images. Our key idea is to first learn a low-resolution version of the image and then generate patches of successively growing resolutions conditioned on previous scales. In a domain translation use-case scenario, 3D thorax CTs of size 512x512x512 and thorax X-rays of size 2048x2048 are generated and we show that, due to the constant GPU memory demand of our method, arbitrarily large images of high resolution can be generated. Moreover, compared to common patch-based approaches, our multi-resolution scheme enables better image quality and prevents patch artifacts.