Improving the Neural Algorithm of Artistic Style
This work addresses visual artifacts in style transfer for users of neural artistic algorithms, but it is incremental as it builds on an existing method.
The paper tackled the problem of the Neural Algorithm of Artistic Style failing to capture complex properties like separate foreground and background styles, leading to visual artifacts, by modifying the style representation to impose tighter constraints, resulting in subjective improvements from barely noticeable to significant in style transfer quality.
In this work we investigate different avenues of improving the Neural Algorithm of Artistic Style (by Leon A. Gatys, Alexander S. Ecker and Matthias Bethge, arXiv:1508.06576). While showing great results when transferring homogeneous and repetitive patterns, the original style representation often fails to capture more complex properties, like having separate styles of foreground and background. This leads to visual artifacts and undesirable textures appearing in unexpected regions when performing style transfer. We tackle this issue with a variety of approaches, mostly by modifying the style representation in order for it to capture more information and impose a tighter constraint on the style transfer result. In our experiments, we subjectively evaluate our best method as producing from barely noticeable to significant improvements in the quality of style transfer.