Balanced Image Stylization with Style Matching Score
This work addresses a long-standing problem in image stylization for applications in computer vision and graphics, offering an incremental improvement with novel techniques.
The paper tackles the challenge of balancing style transfer and content preservation in image stylization by introducing Style Matching Score (SMS), a novel optimization method that reframes stylization as a style distribution matching problem, resulting in outperforming state-of-the-art approaches as verified by extensive experiments.
We present Style Matching Score (SMS), a novel optimization method for image stylization with diffusion models. Balancing effective style transfer with content preservation is a long-standing challenge. Unlike existing efforts, our method reframes image stylization as a style distribution matching problem. The target style distribution is estimated from off-the-shelf style-dependent LoRAs via carefully designed score functions. To preserve content information adaptively, we propose Progressive Spectrum Regularization, which operates in the frequency domain to guide stylization progressively from low-frequency layouts to high-frequency details. In addition, we devise a Semantic-Aware Gradient Refinement technique that leverages relevance maps derived from diffusion semantic priors to selectively stylize semantically important regions. The proposed optimization formulation extends stylization from pixel space to parameter space, readily applicable to lightweight feedforward generators for efficient one-step stylization. SMS effectively balances style alignment and content preservation, outperforming state-of-the-art approaches, verified by extensive experiments.