LUCSS: Language-based User-customized Colourization of Scene Sketches
This addresses the problem of customizing colorization for scene sketches for users in digital art or design, but it is incremental as it builds on existing segmentation and captioning methods.
The paper tackles the problem of interactive colorization of scene sketches by introducing LUCSS, a language-based system that uses deep neural networks to segment sketches, generate captions, and allow user edits for colorization, with experiments showing its effectiveness and desirability over alternatives.
We introduce LUCSS, a language-based system for interactive col- orization of scene sketches, based on their semantic understanding. LUCSS is built upon deep neural networks trained via a large-scale repository of scene sketches and cartoon-style color images with text descriptions. It con- sists of three sequential modules. First, given a scene sketch, the segmenta- tion module automatically partitions an input sketch into individual object instances. Next, the captioning module generates the text description with spatial relationships based on the instance-level segmentation results. Fi- nally, the interactive colorization module allows users to edit the caption and produce colored images based on the altered caption. Our experiments show the effectiveness of our approach and the desirability of its compo- nents to alternative choices.