Towards Arbitrary Text-driven Image Manipulation via Space Alignment
This work addresses the need for more efficient and flexible image editing tools for users, though it is incremental as it builds on existing GAN inversion and CLIP methods.
The authors tackled the problem of inefficient text-driven image manipulation by proposing a framework that aligns CLIP and StyleGAN spaces, enabling arbitrary editing modes without extra optimization costs, achieving real-time results and superior performance in experiments.
The recent GAN inversion methods have been able to successfully invert the real image input to the corresponding editable latent code in StyleGAN. By combining with the language-vision model (CLIP), some text-driven image manipulation methods are proposed. However, these methods require extra costs to perform optimization for a certain image or a new attribute editing mode. To achieve a more efficient editing method, we propose a new Text-driven image Manipulation framework via Space Alignment (TMSA). The Space Alignment module aims to align the same semantic regions in CLIP and StyleGAN spaces. Then, the text input can be directly accessed into the StyleGAN space and be used to find the semantic shift according to the text description. The framework can support arbitrary image editing mode without additional cost. Our work provides the user with an interface to control the attributes of a given image according to text input and get the result in real time. Ex tensive experiments demonstrate our superior performance over prior works.