Whitening and Coloring batch transform for GANs
This addresses training stability and conditioning quality for GAN users, but is incremental as it builds on existing normalization techniques.
The paper tackles the problem of training instability in GANs and limited conditioning in cGANs by proposing a Whitening and Coloring batch transform, which generalizes Batch Normalization and conditional Batch Normalization. The result is consistent improvements across datasets and GAN frameworks, with CIFAR-10 conditioned results surpassing all previous works.
Batch Normalization (BN) is a common technique used to speed-up and stabilize training. On the other hand, the learnable parameters of BN are commonly used in conditional Generative Adversarial Networks (cGANs) for representing class-specific information using conditional Batch Normalization (cBN). In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization. We show that our conditional Coloring can represent categorical conditioning information which largely helps the cGAN qualitative results. Moreover, we show that full-feature whitening is important in a general GAN scenario in which the training process is known to be highly unstable. We test our approach on different datasets and using different GAN networks and training protocols, showing a consistent improvement in all the tested frameworks. Our CIFAR-10 conditioned results are higher than all previous works on this dataset.