Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer
This addresses the problem of computational inefficiency and content loss in diffusion-based style transfer for users in image editing and generation, offering an incremental improvement over existing methods.
The paper tackles the trade-off between style transformation and content preservation in text-guided diffusion image style transfer by proposing a zero-shot contrastive loss that eliminates the need for fine-tuning or auxiliary networks, achieving superior performance in style transfer, image-to-image translation, and manipulation without additional training.
Diffusion models have shown great promise in text-guided image style transfer, but there is a trade-off between style transformation and content preservation due to their stochastic nature. Existing methods require computationally expensive fine-tuning of diffusion models or additional neural network. To address this, here we propose a zero-shot contrastive loss for diffusion models that doesn't require additional fine-tuning or auxiliary networks. By leveraging patch-wise contrastive loss between generated samples and original image embeddings in the pre-trained diffusion model, our method can generate images with the same semantic content as the source image in a zero-shot manner. Our approach outperforms existing methods while preserving content and requiring no additional training, not only for image style transfer but also for image-to-image translation and manipulation. Our experimental results validate the effectiveness of our proposed method.