Imbalance in Balance: Online Concept Balancing in Generation Models
This addresses a specific issue in visual generation for researchers and practitioners, offering an incremental improvement with minimal code changes.
The paper tackles the problem of unstable and error-prone concept responses in visual generation models by proposing an online concept balancing method with a concept-wise equalization loss function, which significantly enhances baseline models' concept response capability on new and public benchmarks.
In visual generation tasks, the responses and combinations of complex concepts often lack stability and are error-prone, which remains an under-explored area. In this paper, we attempt to explore the causal factors for poor concept responses through elaborately designed experiments. We also design a concept-wise equalization loss function (IMBA loss) to address this issue. Our proposed method is online, eliminating the need for offline dataset processing, and requires minimal code changes. In our newly proposed complex concept benchmark Inert-CompBench and two other public test sets, our method significantly enhances the concept response capability of baseline models and yields highly competitive results with only a few codes released at https://github.com/KwaiVGI/IMBA-Loss.