Harmonic Unpaired Image-to-image Translation
This work addresses the challenge of producing high-quality translations without paired data, which is crucial for applications like medical imaging and object transfiguration, representing a strong incremental advance over existing methods like CycleGAN.
The paper tackles the problem of artifacts and degenerated transformations in unpaired image-to-image translation by introducing a smoothness term over the sample graph to enforce consistent mappings, resulting in significant qualitative and quantitative improvements, such as halving the mean-squared error in a medical imaging task and generating images preferred by radiologists in 95% of cases.
The recent direction of unpaired image-to-image translation is on one hand very exciting as it alleviates the big burden in obtaining label-intensive pixel-to-pixel supervision, but it is on the other hand not fully satisfactory due to the presence of artifacts and degenerated transformations. In this paper, we take a manifold view of the problem by introducing a smoothness term over the sample graph to attain harmonic functions to enforce consistent mappings during the translation. We develop HarmonicGAN to learn bi-directional translations between the source and the target domains. With the help of similarity-consistency, the inherent self-consistency property of samples can be maintained. Distance metrics defined on two types of features including histogram and CNN are exploited. Under an identical problem setting as CycleGAN, without additional manual inputs and only at a small training-time cost, HarmonicGAN demonstrates a significant qualitative and quantitative improvement over the state of the art, as well as improved interpretability. We show experimental results in a number of applications including medical imaging, object transfiguration, and semantic labeling. We outperform the competing methods in all tasks, and for a medical imaging task in particular our method turns CycleGAN from a failure to a success, halving the mean-squared error, and generating images that radiologists prefer over competing methods in 95% of cases.