SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization
This work addresses limitations in style transfer for computer vision applications, offering an incremental improvement over prior methods.
The paper tackles the problem of arbitrary artistic style transfer by proposing SAFIN, a plug-and-play module combining self-attention and normalization, which enhances stylization when integrated into existing methods and reduces unwanted textures in a novel multi-scale network.
Artistic style transfer aims to transfer the style characteristics of one image onto another image while retaining its content. Existing approaches commonly leverage various normalization techniques, although these face limitations in adequately transferring diverse textures to different spatial locations. Self-Attention-based approaches have tackled this issue with partial success but suffer from unwanted artifacts. Motivated by these observations, this paper aims to combine the best of both worlds: self-attention and normalization. That yields a new plug-and-play module that we name Self-Attentive Factorized Instance Normalization (SAFIN). SAFIN is essentially a spatially adaptive normalization module whose parameters are inferred through attention on the content and style image. We demonstrate that plugging SAFIN into the base network of another state-of-the-art method results in enhanced stylization. We also develop a novel base network composed of Wavelet Transform for multi-scale style transfer, which when combined with SAFIN, produces visually appealing results with lesser unwanted textures.