MeVGAN: GAN-based Plugin Model for Video Generation with Applications in Colonoscopy
This work addresses the need for efficient video generation in medical applications like colonoscopy simulators, but it is incremental as it builds on existing GAN methods with a plugin approach.
The authors tackled the problem of high-resolution video generation, which is memory-intensive, by proposing MeVGAN, a GAN-based plugin model that uses a pre-trained 2D-image GAN with a simple neural network to generate trajectories in noise space for video creation. They applied it to generate synthetic colonoscopy videos, showing it can produce good quality videos for potential use in virtual simulators.
Video generation is important, especially in medicine, as much data is given in this form. However, video generation of high-resolution data is a very demanding task for generative models, due to the large need for memory. In this paper, we propose Memory Efficient Video GAN (MeVGAN) - a Generative Adversarial Network (GAN) which uses plugin-type architecture. We use a pre-trained 2D-image GAN and only add a simple neural network to construct respective trajectories in the noise space, so that the trajectory forwarded through the GAN model constructs a real-life video. We apply MeVGAN in the task of generating colonoscopy videos. Colonoscopy is an important medical procedure, especially beneficial in screening and managing colorectal cancer. However, because colonoscopy is difficult and time-consuming to learn, colonoscopy simulators are widely used in educating young colonoscopists. We show that MeVGAN can produce good quality synthetic colonoscopy videos, which can be potentially used in virtual simulators.