High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network
This addresses the computational bottleneck for real-time high-resolution image translation tasks, though it is incremental as it builds on existing I2IT methods with efficiency improvements.
The paper tackles the problem of slow high-resolution photorealistic image-to-image translation by proposing a Laplacian Pyramid Translation Network (LPTN), which achieves real-time translation of 4K images on a normal GPU while maintaining comparable performance to existing methods.
Existing image-to-image translation (I2IT) methods are either constrained to low-resolution images or long inference time due to their heavy computational burden on the convolution of high-resolution feature maps. In this paper, we focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. Specifically, we reveal that the attribute transformations, such as illumination and color manipulation, relate more to the low-frequency component, while the content details can be adaptively refined on high-frequency components. We consequently propose a Laplacian Pyramid Translation Network (LPTN) to simultaneously perform these two tasks, where we design a lightweight network for translating the low-frequency component with reduced resolution and a progressive masking strategy to efficiently refine the high-frequency ones. Our model avoids most of the heavy computation consumed by processing high-resolution feature maps and faithfully preserves the image details. Extensive experimental results on various tasks demonstrate that the proposed method can translate 4K images in real-time using one normal GPU while achieving comparable transformation performance against existing methods. Datasets and codes are available: https://github.com/csjliang/LPTN.