Generalized Transformation-based Gradient
This work addresses a bottleneck in variational inference for researchers and practitioners, offering an incremental improvement by extending an existing method.
The paper tackles the limitation of the reparameterization trick in variational inference, which is restricted to distributions with tractable inverse CDFs, by generalizing it to allow any transformation, and shows that this model is a special case of control variates, enabling combination of their advantages.
The reparameterization trick has become one of the most useful tools in the field of variational inference. However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution functions or are expressible as deterministic transformations of such distributions. In this paper, we generalized the reparameterization trick by allowing a general transformation. We discover that the proposed model is a special case of control variate indicating that the proposed model can combine the advantages of CV and generalized reparameterization.