Calibri: Enhancing Diffusion Transformers via Parameter-Efficient Calibration
This work addresses the challenge of optimizing generative AI models efficiently, though it appears incremental as it builds on existing DiT frameworks with minor modifications.
The paper tackled the problem of enhancing Diffusion Transformers (DiTs) for generative tasks by introducing a parameter-efficient calibration method, resulting in improved performance across text-to-image models and reduced inference steps while maintaining high-quality outputs.
In this paper, we uncover the hidden potential of Diffusion Transformers (DiTs) to significantly enhance generative tasks. Through an in-depth analysis of the denoising process, we demonstrate that introducing a single learned scaling parameter can significantly improve the performance of DiT blocks. Building on this insight, we propose Calibri, a parameter-efficient approach that optimally calibrates DiT components to elevate generative quality. Calibri frames DiT calibration as a black-box reward optimization problem, which is efficiently solved using an evolutionary algorithm and modifies just ~100 parameters. Experimental results reveal that despite its lightweight design, Calibri consistently improves performance across various text-to-image models. Notably, Calibri also reduces the inference steps required for image generation, all while maintaining high-quality outputs.