Resampling Gradients Vanish in Differentiable Sequential Monte Carlo Samplers
This addresses a specific issue in differentiable sampling methods for researchers in machine learning, offering an incremental improvement.
The paper tackles the problem of low effective sample size in Differentiable Annealed Importance Sampling (DAIS) by proposing an extension with a resampling step inspired by Sequential Monte Carlo, and finds that differentiating through this step is unnecessary, avoiding gradient variance issues.
Annealed Importance Sampling (AIS) moves particles along a Markov chain from a tractable initial distribution to an intractable target distribution. The recently proposed Differentiable AIS (DAIS) (Geffner and Domke, 2021; Zhang et al., 2021) enables efficient optimization of the transition kernels of AIS and of the distributions. However, we observe a low effective sample size in DAIS, indicating degenerate distributions. We thus propose to extend DAIS by a resampling step inspired by Sequential Monte Carlo. Surprisingly, we find empirically-and can explain theoretically-that it is not necessary to differentiate through the resampling step which avoids gradient variance issues observed in similar approaches for Particle Filters (Maddison et al., 2017; Naesseth et al., 2018; Le et al., 2018).