Data-Efficient Brushstroke Generation with Diffusion Models for Oil Painting
This addresses data scarcity in creative multimedia systems for artists and designers, offering a data-efficient method for controllable brushstroke generation, though it is incremental as it builds on diffusion models.
The paper tackled the problem of generating human-like brushstrokes from a small dataset of 470 hand-drawn samples, proposing StrokeDiff with Smooth Regularization to stabilize training and achieve diverse, structurally coherent brushstrokes that enable paintings with richer texture and layering.
Many creative multimedia systems are built upon visual primitives such as strokes or textures, which are difficult to collect at scale and fundamentally different from natural image data. This data scarcity makes it challenging for modern generative models to learn expressive and controllable primitives, limiting their use in process-aware content creation. In this work, we study the problem of learning human-like brushstroke generation from a small set of hand-drawn samples (n=470) and propose StrokeDiff, a diffusion-based framework with Smooth Regularization (SmR). SmR injects stochastic visual priors during training, providing a simple mechanism to stabilize diffusion models under sparse supervision without altering the inference process. We further show how the learned primitives can be made controllable through a Bézier-based conditioning module and integrated into a complete stroke-based painting pipeline, including prediction, generation, ordering, and compositing. This demonstrates how data-efficient primitive modeling can support expressive and structured multimedia content creation. Experiments indicate that the proposed approach produces diverse and structurally coherent brushstrokes and enables paintings with richer texture and layering, validated by both automatic metrics and human evaluation.