QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity
This addresses the issue of generating realistic artworks in style transfer, though it appears incremental as it builds on existing loss-based approaches.
The paper tackles the problem of low visual fidelity in style transfer by introducing QuantArt, a framework that uses vector quantization to align generated artworks with real artwork distributions, achieving significantly higher visual fidelity compared to existing methods.
The mechanism of existing style transfer algorithms is by minimizing a hybrid loss function to push the generated image toward high similarities in both content and style. However, this type of approach cannot guarantee visual fidelity, i.e., the generated artworks should be indistinguishable from real ones. In this paper, we devise a new style transfer framework called QuantArt for high visual-fidelity stylization. QuantArt pushes the latent representation of the generated artwork toward the centroids of the real artwork distribution with vector quantization. By fusing the quantized and continuous latent representations, QuantArt allows flexible control over the generated artworks in terms of content preservation, style similarity, and visual fidelity. Experiments on various style transfer settings show that our QuantArt framework achieves significantly higher visual fidelity compared with the existing style transfer methods.