GILBO: One Metric to Measure Them All
This provides a universal metric for evaluating generative models like VAEs and GANs, though it is incremental as it builds on existing information theory concepts.
The authors introduced GILBO, a tractable lower bound on mutual information to measure the complexity of latent variable generative models, and computed it for 800 models across four datasets to analyze results.
We propose a simple, tractable lower bound on the mutual information contained in the joint generative density of any latent variable generative model: the GILBO (Generative Information Lower BOund). It offers a data-independent measure of the complexity of the learned latent variable description, giving the log of the effective description length. It is well-defined for both VAEs and GANs. We compute the GILBO for 800 GANs and VAEs each trained on four datasets (MNIST, FashionMNIST, CIFAR-10 and CelebA) and discuss the results.