Moment- and Power-Spectrum-Based Gaussianity Regularization for Text-to-Image Models
This work addresses the challenge of improving optimization in the latent space of text-to-image models for downstream tasks, representing an incremental advancement in Gaussianity regularization techniques.
The authors tackled the problem of enforcing standard Gaussianity in text-to-image models by proposing a novel regularization loss that combines moment-based and power spectrum-based components, which outperforms previous methods in preventing reward hacking and accelerating convergence for tasks like enhancing aesthetics and text alignment.
We propose a novel regularization loss that enforces standard Gaussianity, encouraging samples to align with a standard Gaussian distribution. This facilitates a range of downstream tasks involving optimization in the latent space of text-to-image models. We treat elements of a high-dimensional sample as one-dimensional standard Gaussian variables and define a composite loss that combines moment-based regularization in the spatial domain with power spectrum-based regularization in the spectral domain. Since the expected values of moments and power spectrum distributions are analytically known, the loss promotes conformity to these properties. To ensure permutation invariance, the losses are applied to randomly permuted inputs. Notably, existing Gaussianity-based regularizations fall within our unified framework: some correspond to moment losses of specific orders, while the previous covariance-matching loss is equivalent to our spectral loss but incurs higher time complexity due to its spatial-domain computation. We showcase the application of our regularization in generative modeling for test-time reward alignment with a text-to-image model, specifically to enhance aesthetics and text alignment. Our regularization outperforms previous Gaussianity regularization, effectively prevents reward hacking and accelerates convergence.