Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Model
This addresses a specific problem in layout generation for designers using DDMs, offering an incremental improvement by enhancing existing models.
The paper tackles the layout sticking phenomenon in discrete diffusion models (DDMs) for layout generation, where models struggle to correct inharmonious layouts after generation, and proposes Layout-Corrector, a layout-assessment module that identifies and reinitializes erroneous tokens during generation. The method consistently boosts performance on common benchmarks when combined with various state-of-the-art DDMs, significantly mitigating performance drops in fast sampling.
Layout generation is a task to synthesize a harmonious layout with elements characterized by attributes such as category, position, and size. Human designers experiment with the placement and modification of elements to create aesthetic layouts, however, we observed that current discrete diffusion models (DDMs) struggle to correct inharmonious layouts after they have been generated. In this paper, we first provide novel insights into layout sticking phenomenon in DDMs and then propose a simple yet effective layout-assessment module Layout-Corrector, which works in conjunction with existing DDMs to address the layout sticking problem. We present a learning-based module capable of identifying inharmonious elements within layouts, considering overall layout harmony characterized by complex composition. During the generation process, Layout-Corrector evaluates the correctness of each token in the generated layout, reinitializing those with low scores to the ungenerated state. The DDM then uses the high-scored tokens as clues to regenerate the harmonized tokens. Layout-Corrector, tested on common benchmarks, consistently boosts layout-generation performance when in conjunction with various state-of-the-art DDMs. Furthermore, our extensive analysis demonstrates that the Layout-Corrector (1) successfully identifies erroneous tokens, (2) facilitates control over the fidelity-diversity trade-off, and (3) significantly mitigates the performance drop associated with fast sampling.