LayerFusion: Harmonized Multi-Layer Text-to-Image Generation with Generative Priors
This addresses a need in creative workflows for graphic design, animation, and digital art, where layer-based editing is essential, but it is incremental as it builds on existing diffusion models.
The paper tackles the problem of generating layered content like transparent images with foreground and background layers in text-to-image generation, achieving significant improvements in visual coherence, image quality, and layer consistency compared to baselines.
Large-scale diffusion models have achieved remarkable success in generating high-quality images from textual descriptions, gaining popularity across various applications. However, the generation of layered content, such as transparent images with foreground and background layers, remains an under-explored area. Layered content generation is crucial for creative workflows in fields like graphic design, animation, and digital art, where layer-based approaches are fundamental for flexible editing and composition. In this paper, we propose a novel image generation pipeline based on Latent Diffusion Models (LDMs) that generates images with two layers: a foreground layer (RGBA) with transparency information and a background layer (RGB). Unlike existing methods that generate these layers sequentially, our approach introduces a harmonized generation mechanism that enables dynamic interactions between the layers for more coherent outputs. We demonstrate the effectiveness of our method through extensive qualitative and quantitative experiments, showing significant improvements in visual coherence, image quality, and layer consistency compared to baseline methods.