Content-aware Tile Generation using Exterior Boundary Inpainting
This addresses the need for automated, content-aware tile generation in graphics and design, offering a novel approach that enhances diversity and continuity without requiring custom model training.
The paper tackles the problem of generating diverse sets of mutually tileable images by using a learning-based method that leverages pretrained diffusion models for inpainting based on exterior boundary conditions and text prompts, demonstrating flexibility across tiling schemes like Wang tiles and introducing a Dual Wang tiling scheme for improved texture continuity and diversity.
We present a novel and flexible learning-based method for generating tileable image sets. Our method goes beyond simple self-tiling, supporting sets of mutually tileable images that exhibit a high degree of diversity. To promote diversity we decouple structure from content by foregoing explicit copying of patches from an exemplar image. Instead we leverage the prior knowledge of natural images and textures embedded in large-scale pretrained diffusion models to guide tile generation constrained by exterior boundary conditions and a text prompt to specify the content. By carefully designing and selecting the exterior boundary conditions, we can reformulate the tile generation process as an inpainting problem, allowing us to directly employ existing diffusion-based inpainting models without the need to retrain a model on a custom training set. We demonstrate the flexibility and efficacy of our content-aware tile generation method on different tiling schemes, such as Wang tiles, from only a text prompt. Furthermore, we introduce a novel Dual Wang tiling scheme that provides greater texture continuity and diversity than existing Wang tile variants.