HyperNST: Hyper-Networks for Neural Style Transfer
This work addresses the challenge of improving content preservation in neural style transfer for artistic applications, particularly in portrait stylization, representing an incremental advancement in the field.
The paper tackles the problem of artistic stylization of images by introducing HyperNST, a neural style transfer method based on hyper-networks and StyleGAN2, which uses a pre-trained metric space for style-based visual search to drive style transfer and interpolation. The result shows that HyperNST exceeds state-of-the-art in content preservation for stylized portraits while maintaining good style transfer performance.
We present HyperNST; a neural style transfer (NST) technique for the artistic stylization of images, based on Hyper-networks and the StyleGAN2 architecture. Our contribution is a novel method for inducing style transfer parameterized by a metric space, pre-trained for style-based visual search (SBVS). We show for the first time that such space may be used to drive NST, enabling the application and interpolation of styles from an SBVS system. The technical contribution is a hyper-network that predicts weight updates to a StyleGAN2 pre-trained over a diverse gamut of artistic content (portraits), tailoring the style parameterization on a per-region basis using a semantic map of the facial regions. We show HyperNST to exceed state of the art in content preservation for our stylized content while retaining good style transfer performance.