Style Transfer With Adaptation to the Central Objects of the Scene
This work addresses a specific issue in style transfer for image processing, making it more practical for applications like photo editing, but it is incremental as it builds on existing methods.
The paper tackles the problem of style transfer causing central objects like faces or text to become unrecognizable by proposing an algorithm that detects these objects and applies style non-uniformly, resulting in higher quality stylization as demonstrated by qualitative and user evaluations.
Style transfer is a problem of rendering image with some content in the style of another image, for example a family photo in the style of a painting of some famous artist. The drawback of classical style transfer algorithm is that it imposes style uniformly on all parts of the content image, which perturbs central objects on the content image, such as faces or text, and makes them unrecognizable. This work proposes a novel style transfer algorithm which automatically detects central objects on the content image, generates spatial importance mask and imposes style non-uniformly: central objects are stylized less to preserve their recognizability and other parts of the image are stylized as usual to preserve the style. Three methods of automatic central object detection are proposed and evaluated qualitatively and via a user evaluation study. Both comparisons demonstrate higher quality of stylization compared to the classical style transfer method.