HSI: A Holistic Style Injector for Arbitrary Style Transfer
This work improves style transfer for image synthesis applications, but it is incremental as it builds on existing attention-based methods.
The paper tackles the problem of arbitrary style transfer by addressing the limitations of attention-based methods, which overly focus on local patterns and have high computational complexity, and proposes HSI to achieve better style fidelity and efficiency, outperforming state-of-the-art approaches in both effectiveness and efficiency.
Attention-based arbitrary style transfer methods have gained significant attention recently due to their impressive ability to synthesize style details. However, the point-wise matching within the attention mechanism may overly focus on local patterns such that neglect the remarkable global features of style images. Additionally, when processing large images, the quadratic complexity of the attention mechanism will bring high computational load. To alleviate above problems, we propose Holistic Style Injector (HSI), a novel attention-style transformation module to deliver artistic expression of target style. Specifically, HSI performs stylization only based on global style representation that is more in line with the characteristics of style transfer, to avoid generating local disharmonious patterns in stylized images. Moreover, we propose a dual relation learning mechanism inside the HSI to dynamically render images by leveraging semantic similarity in content and style, ensuring the stylized images preserve the original content and improve style fidelity. Note that the proposed HSI achieves linear computational complexity because it establishes feature mapping through element-wise multiplication rather than matrix multiplication. Qualitative and quantitative results demonstrate that our method outperforms state-of-the-art approaches in both effectiveness and efficiency.