Quantized Variational Inference
This work addresses computational efficiency and gradient stability in variational inference, a key method in machine learning for approximate Bayesian inference, though it appears incremental as it builds on existing techniques like Voronoi tesselation and extrapolation.
The paper tackles the problem of optimizing the Evidence Lower Bound in variational inference by introducing Quantized Variational Inference, which uses Optimal Voronoi Tesselation to achieve variance-free gradients with asymptotically decaying bias, and applies Richardson extrapolation to improve asymptotic bounds, resulting in fast convergence for gradient estimators at comparable computational cost.
We present Quantized Variational Inference, a new algorithm for Evidence Lower Bound maximization. We show how Optimal Voronoi Tesselation produces variance free gradients for ELBO optimization at the cost of introducing asymptotically decaying bias. Subsequently, we propose a Richardson extrapolation type method to improve the asymptotic bound. We show that using the Quantized Variational Inference framework leads to fast convergence for both score function and the reparametrized gradient estimator at a comparable computational cost. Finally, we propose several experiments to assess the performance of our method and its limitations.