PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation
This addresses the problem of generating realistic images from different viewpoints for applications like robotics or AR, but it appears incremental as it builds on existing GAN-based translation methods.
The paper tackles cross-view image translation with minimal overlap between source and target views by proposing PI-Trans, a GAN that uses Parallel-ConvMLP and Implicit Transformation modules, achieving the best qualitative and quantitative performance by a large margin on two challenging datasets.
For semantic-guided cross-view image translation, it is crucial to learn where to sample pixels from the source view image and where to reallocate them guided by the target view semantic map, especially when there is little overlap or drastic view difference between the source and target images. Hence, one not only needs to encode the long-range dependencies among pixels in both the source view image and target view semantic map but also needs to translate these learned dependencies. To this end, we propose a novel generative adversarial network, PI-Trans, which mainly consists of a novel Parallel-ConvMLP module and an Implicit Transformation module at multiple semantic levels. Extensive experimental results show that PI-Trans achieves the best qualitative and quantitative performance by a large margin compared to the state-of-the-art methods on two challenging datasets. The source code is available at https://github.com/Amazingren/PI-Trans.