FastCLIPstyler: Optimisation-free Text-based Image Style Transfer Using Style Representations
This work addresses the practical limitation of slow optimization in language-driven style transfer for applications requiring real-time or edge device use.
The paper tackles the slow runtime of text-based image style transfer by introducing FastCLIPstyler, which stylizes images in a single forward pass, and EdgeCLIPstyler for resource-constrained devices, achieving superior stylization quality and significantly improved efficiency.
In recent years, language-driven artistic style transfer has emerged as a new type of style transfer technique, eliminating the need for a reference style image by using natural language descriptions of the style. The first model to achieve this, called CLIPstyler, has demonstrated impressive stylisation results. However, its lengthy optimisation procedure at runtime for each query limits its suitability for many practical applications. In this work, we present FastCLIPstyler, a generalised text-based image style transfer model capable of stylising images in a single forward pass for arbitrary text inputs. Furthermore, we introduce EdgeCLIPstyler, a lightweight model designed for compatibility with resource-constrained devices. Through quantitative and qualitative comparisons with state-of-the-art approaches, we demonstrate that our models achieve superior stylisation quality based on measurable metrics while offering significantly improved runtime efficiency, particularly on edge devices.