Two Methods For Wild Variational Inference
This work addresses a bottleneck in variational inference for researchers and practitioners by allowing more flexible inference network designs, though it appears incremental as it builds on existing methods like SGLD.
The paper tackles the limitation of variational inference requiring tractable density functions by introducing wild variational inference methods that relax this constraint, enabling application to more challenging cases like automatically adjusting step sizes in stochastic gradient Langevin dynamics (SGLD) and yielding significant improvements over hand-designed schemes.
Variational inference provides a powerful tool for approximate probabilistic in- ference on complex, structured models. Typical variational inference methods, however, require to use inference networks with computationally tractable proba- bility density functions. This largely limits the design and implementation of vari- ational inference methods. We consider wild variational inference methods that do not require tractable density functions on the inference networks, and hence can be applied in more challenging cases. As an example of application, we treat stochastic gradient Langevin dynamics (SGLD) as an inference network, and use our methods to automatically adjust the step sizes of SGLD, yielding significant improvement over the hand-designed step size schemes