Text2Layer: Layered Image Generation using Latent Diffusion Model
This enables better compositing workflows for image editing, though it is incremental in applying diffusion models to a specific domain.
The paper tackles layered image generation by training an autoencoder and diffusion models to produce background, foreground, mask, and composed image simultaneously, resulting in high-quality layered images and masks.
Layer compositing is one of the most popular image editing workflows among both amateurs and professionals. Motivated by the success of diffusion models, we explore layer compositing from a layered image generation perspective. Instead of generating an image, we propose to generate background, foreground, layer mask, and the composed image simultaneously. To achieve layered image generation, we train an autoencoder that is able to reconstruct layered images and train diffusion models on the latent representation. One benefit of the proposed problem is to enable better compositing workflows in addition to the high-quality image output. Another benefit is producing higher-quality layer masks compared to masks produced by a separate step of image segmentation. Experimental results show that the proposed method is able to generate high-quality layered images and initiates a benchmark for future work.