WikiStyle+: A Multimodal Approach to Content-Style Representation Disentanglement for Artistic Image Stylization
This addresses the issue of content leakage and limited modality support in artistic image stylization for applications in digital art and media, representing an incremental improvement over existing methods.
The paper tackles the problem of incomplete content-style disentanglement in artistic image stylization by proposing a multimodal approach using a new dataset and a disentangled representations-guided diffusion model, achieving thorough disentanglement and more refined stylization.
Artistic image stylization aims to render the content provided by text or image with the target style, where content and style decoupling is the key to achieve satisfactory results. However, current methods for content and style disentanglement primarily rely on image supervision, which leads to two problems: 1) models can only support one modality for style or content input;2) incomplete disentanglement resulting in content leakage from the reference image. To address the above issues, this paper proposes a multimodal approach to content-style disentanglement for artistic image stylization. We construct a \textit{WikiStyle+} dataset consists of artworks with corresponding textual descriptions for style and content. Based on the multimodal dataset, we propose a disentangled representations-guided diffusion model. The disentangled representations are first learned by Q-Formers and then injected into a pre-trained diffusion model using learnable multi-step cross-attention layers. Experimental results show that our method achieves a thorough disentanglement of content and style in reference images under multimodal supervision, thereby enabling more refined stylization that aligns with the artistic characteristics of the reference style. The code of our method will be available upon acceptance.