ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variables
This addresses a bottleneck in training discrete latent variable models for researchers and practitioners, offering an incremental improvement over existing gradient estimators like DisARM and LOORF.
The paper tackles the problem of gradient estimation for binary variables in discrete latent variable models by proposing ARMS, an antithetic REINFORCE-based multi-sample gradient estimator that uses a copula to generate mutually antithetic samples, and experimental results show it outperforms competing methods on several datasets for training generative models.
Estimating the gradients for binary variables is a task that arises frequently in various domains, such as training discrete latent variable models. What has been commonly used is a REINFORCE based Monte Carlo estimation method that uses either independent samples or pairs of negatively correlated samples. To better utilize more than two samples, we propose ARMS, an Antithetic REINFORCE-based Multi-Sample gradient estimator. ARMS uses a copula to generate any number of mutually antithetic samples. It is unbiased, has low variance, and generalizes both DisARM, which we show to be ARMS with two samples, and the leave-one-out REINFORCE (LOORF) estimator, which is ARMS with uncorrelated samples. We evaluate ARMS on several datasets for training generative models, and our experimental results show that it outperforms competing methods. We also develop a version of ARMS for optimizing the multi-sample variational bound, and show that it outperforms both VIMCO and DisARM. The code is publicly available.