Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss
This addresses the costly and challenging task of line art colorization for artists and designers, representing an incremental improvement with novel components like SECat and a two-step training method.
The authors tackled the problem of automating line art colorization by proposing Tag2Pix, a GAN-based method that uses grayscale line art and color tags as input to produce colored images, achieving improved results as demonstrated through quantitative and qualitative evaluations.
Line art colorization is expensive and challenging to automate. A GAN approach is proposed, called Tag2Pix, of line art colorization which takes as input a grayscale line art and color tag information and produces a quality colored image. First, we present the Tag2Pix line art colorization dataset. A generator network is proposed which consists of convolutional layers to transform the input line art, a pre-trained semantic extraction network, and an encoder for input color information. The discriminator is based on an auxiliary classifier GAN to classify the tag information as well as genuineness. In addition, we propose a novel network structure called SECat, which makes the generator properly colorize even small features such as eyes, and also suggest a novel two-step training method where the generator and discriminator first learn the notion of object and shape and then, based on the learned notion, learn colorization, such as where and how to place which color. We present both quantitative and qualitative evaluations which prove the effectiveness of the proposed method.