Top-N: Equivariant set and graph generation without exchangeability
This addresses the challenge of generating sets and graphs more efficiently for applications like molecule design, though it is an incremental advance over existing generative models.
The paper tackles the problem of one-shot set and graph generation by introducing Top-n creation, a differentiable mechanism that replaces i.i.d. sampling to improve training and performance. It shows improvements such as 15% better reconstruction on SetMNIST, 33% higher object detection on CLEVR, and 74% closer distribution matching on a synthetic dataset.
This work addresses one-shot set and graph generation, and, more specifically, the parametrization of probabilistic decoders that map a vector-shaped prior to a distribution over sets or graphs. Sets and graphs are most commonly generated by first sampling points i.i.d. from a normal distribution, and then processing these points along with the prior vector using Transformer layers or Graph Neural Networks. This architecture is designed to generate exchangeable distributions, i.e., all permutations of the generated outputs are equally likely. We however show that it only optimizes a proxy to the evidence lower bound, which makes it hard to train. We then study equivariance in generative settings and show that non-exchangeable methods can still achieve permutation equivariance. Using this result, we introduce Top-n creation, a differentiable generation mechanism that uses the latent vector to select the most relevant points from a trainable reference set. Top-n can replace i.i.d. generation in any Variational Autoencoder or Generative Adversarial Network. Experimentally, our method outperforms i.i.d. generation by 15% at SetMNIST reconstruction, by 33% at object detection on CLEVR, generates sets that are 74% closer to the true distribution on a synthetic molecule-like dataset, and generates more valid molecules on QM9.