Deep Translation Prior: Test-time Training for Photorealistic Style Transfer
This addresses the limited applicability and poor generalization of existing style transfer methods for unseen images or styles, offering a more flexible solution.
The paper tackles the problem of photorealistic style transfer by proposing Deep Translation Prior (DTP), a framework that uses test-time training on input image pairs with untrained networks, eliminating the need for offline training and achieving better generalization and performance compared to state-of-the-art methods.
Recent techniques to solve photorealistic style transfer within deep convolutional neural networks (CNNs) generally require intensive training from large-scale datasets, thus having limited applicability and poor generalization ability to unseen images or styles. To overcome this, we propose a novel framework, dubbed Deep Translation Prior (DTP), to accomplish photorealistic style transfer through test-time training on given input image pair with untrained networks, which learns an image pair-specific translation prior and thus yields better performance and generalization. Tailored for such test-time training for style transfer, we present novel network architectures, with two sub-modules of correspondence and generation modules, and loss functions consisting of contrastive content, style, and cycle consistency losses. Our framework does not require offline training phase for style transfer, which has been one of the main challenges in existing methods, but the networks are to be solely learned during test-time. Experimental results prove that our framework has a better generalization ability to unseen image pairs and even outperforms the state-of-the-art methods.