The Generalized Reparameterization Gradient
This addresses a bottleneck in variational inference for researchers and practitioners by enabling more flexible and efficient gradient estimation, though it is an incremental improvement over existing methods.
The paper tackles the limitation of the reparameterization gradient, which struggles with non-Gaussian distributions like beta or gamma, by introducing a generalized method that extends it to a wider class of variational distributions using invertible transformations. The result is an effective approach that achieves low-variance gradients with just a single sample, as demonstrated on complex probabilistic models.
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterization gradient fit Gaussian distributions. In this paper, we introduce the generalized reparameterization gradient, a method that extends the reparameterization gradient to a wider class of variational distributions. Generalized reparameterizations use invertible transformations of the latent variables which lead to transformed distributions that weakly depend on the variational parameters. This results in new Monte Carlo gradients that combine reparameterization gradients and score function gradients. We demonstrate our approach on variational inference for two complex probabilistic models. The generalized reparameterization is effective: even a single sample from the variational distribution is enough to obtain a low-variance gradient.