AnimatePainter: A Self-Supervised Rendering Framework for Reconstructing Painting Process
This addresses the need for accessible tools to reconstruct painting processes for artists and educators, though it is incremental as it builds on existing video generation and depth estimation methods.
The paper tackles the problem of generating drawing processes from images without human-annotated data by proposing a self-supervised framework that treats it as video generation, achieving realistic results without real drawing process data.
Humans can intuitively decompose an image into a sequence of strokes to create a painting, yet existing methods for generating drawing processes are limited to specific data types and often rely on expensive human-annotated datasets. We propose a novel self-supervised framework for generating drawing processes from any type of image, treating the task as a video generation problem. Our approach reverses the drawing process by progressively removing strokes from a reference image, simulating a human-like creation sequence. Crucially, our method does not require costly datasets of real human drawing processes; instead, we leverage depth estimation and stroke rendering to construct a self-supervised dataset. We model human drawings as "refinement" and "layering" processes and introduce depth fusion layers to enable video generation models to learn and replicate human drawing behavior. Extensive experiments validate the effectiveness of our approach, demonstrating its ability to generate realistic drawings without the need for real drawing process data.