Inverse Painting: Reconstructing The Painting Process
This work addresses the problem of understanding the artistic creation process for art historians and enthusiasts, offering a novel way to visualize painting evolution.
This paper reconstructs a time-lapse video of how a given painting may have been created by formulating it as an autoregressive image generation problem. The model learns from real painting videos and uses text and region understanding to define painting instructions, updating the canvas with a novel diffusion-based renderer.
Given an input painting, we reconstruct a time-lapse video of how it may have been painted. We formulate this as an autoregressive image generation problem, in which an initially blank "canvas" is iteratively updated. The model learns from real artists by training on many painting videos. Our approach incorporates text and region understanding to define a set of painting "instructions" and updates the canvas with a novel diffusion-based renderer. The method extrapolates beyond the limited, acrylic style paintings on which it has been trained, showing plausible results for a wide range of artistic styles and genres.