Structured Pattern Expansion with Diffusion Models
This work addresses the need for more controllable pattern synthesis in digital asset creation, representing an incremental improvement with domain-specific applications.
The paper tackled the problem of synthesizing structured, stationary patterns using diffusion models, which are less reliable and controllable in this domain, by developing a method that expands partially hand-drawn patterns into larger designs while preserving structure and details, resulting in outperformance of existing models in generating diverse and consistent patterns.
Recent advances in diffusion models have significantly improved the synthesis of materials, textures, and 3D shapes. By conditioning these models via text or images, users can guide the generation, reducing the time required to create digital assets. In this paper, we address the synthesis of structured, stationary patterns, where diffusion models are generally less reliable and, more importantly, less controllable. Our approach leverages the generative capabilities of diffusion models specifically adapted for the pattern domain. It enables users to exercise direct control over the synthesis by expanding a partially hand-drawn pattern into a larger design while preserving the structure and details of the input. To enhance pattern quality, we fine-tune an image-pretrained diffusion model on structured patterns using Low-Rank Adaptation (LoRA), apply a noise rolling technique to ensure tileability, and utilize a patch-based approach to facilitate the generation of large-scale assets. We demonstrate the effectiveness of our method through a comprehensive set of experiments, showing that it outperforms existing models in generating diverse, consistent patterns that respond directly to user input.