Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generation
This work addresses the problem of enhancing generative capabilities in unified models for text-to-image generation, representing an incremental improvement over existing reasoning methods.
The paper tackles the limitation of unified models in text-to-image generation by proposing the Understanding-in-Generation (UiG) framework, which integrates understanding capabilities into the generation process via image editing, resulting in a 3.92% performance gain on the TIIF benchmark.
Recent works have made notable advancements in enhancing unified models for text-to-image generation through the Chain-of-Thought (CoT). However, these reasoning methods separate the processes of understanding and generation, which limits their ability to guide the reasoning of unified models in addressing the deficiencies of their generative capabilities. To this end, we propose a novel reasoning framework for unified models, Understanding-in-Generation (UiG), which harnesses the robust understanding capabilities of unified models to reinforce their performance in image generation. The core insight of our UiG is to integrate generative guidance by the strong understanding capabilities during the reasoning process, thereby mitigating the limitations of generative abilities. To achieve this, we introduce "Image Editing" as a bridge to infuse understanding into the generation process. Initially, we verify the generated image and incorporate the understanding of unified models into the editing instructions. Subsequently, we enhance the generated image step by step, gradually infusing the understanding into the generation process. Our UiG framework demonstrates a significant performance improvement in text-to-image generation over existing text-to-image reasoning methods, e.g., a 3.92% gain on the long prompt setting of the TIIF benchmark. The project code: https://github.com/QC-LY/UiG