CoMoGAN: continuous model-guided image-to-image translation
This work addresses the need for flexible and high-quality image translation in computer vision, though it appears incremental as it builds on existing GAN backbones.
The authors tackled the problem of continuous image-to-image translation by introducing CoMoGAN, which uses a functional manifold and novel layers to disentangle content from position, enabling tasks like timelapse generation and outperforming existing methods on all datasets.
CoMoGAN is a continuous GAN relying on the unsupervised reorganization of the target data on a functional manifold. To that matter, we introduce a new Functional Instance Normalization layer and residual mechanism, which together disentangle image content from position on target manifold. We rely on naive physics-inspired models to guide the training while allowing private model/translations features. CoMoGAN can be used with any GAN backbone and allows new types of image translation, such as cyclic image translation like timelapse generation, or detached linear translation. On all datasets, it outperforms the literature. Our code is available at http://github.com/cv-rits/CoMoGAN .