Leveraging Diffusion Models for Stylization using Multiple Style Images
This work addresses style transfer challenges for image generation, but it appears incremental as it builds on existing diffusion models.
The paper tackles the problem of image style transfer by addressing issues like inaccurate style matching and content-style entanglement, proposing a method that leverages multiple style images and achieves state-of-the-art results.
Recent advances in latent diffusion models have enabled exciting progress in image style transfer. However, several key issues remain. For example, existing methods still struggle to accurately match styles. They are often limited in the number of style images that can be used. Furthermore, they tend to entangle content and style in undesired ways. To address this, we propose leveraging multiple style images which helps better represent style features and prevent content leaking from the style images. We design a method that leverages both image prompt adapters and statistical alignment of the features during the denoising process. With this, our approach is designed such that it can intervene both at the cross-attention and the self-attention layers of the denoising UNet. For the statistical alignment, we employ clustering to distill a small representative set of attention features from the large number of attention values extracted from the style samples. As demonstrated in our experimental section, the resulting method achieves state-of-the-art results for stylization.