Reinterpreting Importance-Weighted Autoencoders
This provides a new theoretical perspective for researchers in variational inference, but it is incremental as it reinterprets existing methods without major empirical gains.
The paper reinterprets importance-weighted autoencoders as optimizing the standard variational lower bound with a more complex distribution, deriving this result formally and presenting a tighter lower bound.
The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribution. We formally derive this result, present a tighter lower bound, and visualize the implicit importance-weighted distribution.