Re:Draw -- Context Aware Translation as a Controllable Method for Artistic Production
This addresses the labor-intensive and costly compromises in artistic production, especially for animation, though it appears incremental as it builds on inpainting and image-to-image translation.
The paper tackles the problem of maintaining consistency and detail in hand-drawn animation, particularly for eyes, by introducing context-aware translation, which eliminates the need for production data and achieves a 95.16% preference over existing methods in a user study.
We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation, respecting simultaneously the original input and contextual relevance -- where existing methods fall short. By doing so, our method opens new avenues for the controllable use of AI within artistic creation, from animation to digital art. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications - eyes serve as a focal point that captures viewer attention and conveys a range of emotions, however, the labor-intensive nature of traditional animation often leads to compromises in the complexity and consistency of eye design. Furthermore, we remove the need for production data for training and introduce a new character recognition method that surpasses existing work by not requiring fine-tuning to specific productions. This proposed use case could help maintain consistency throughout production and unlock bolder and more detailed design choices without the production cost drawbacks. A user study shows context-aware translation is preferred over existing work 95.16% of the time.