Notes on optimal approximations for importance sampling
This work provides theoretical guidance for practitioners using importance sampling in Monte Carlo methods, though it is primarily analytical and does not present empirical results.
The paper derives optimal conditions for constructing function approximations that minimize variance when used as importance sampling estimators in Monte Carlo integration. It shows that the optimal projection differs from the commonly used ℓ1 projection, providing an intuitive explanation for this difference.
In this manuscript, we derive optimal conditions for building function approximations that minimize variance when used as importance sampling estimators for Monte Carlo integration problems. Particularly, we study the problem of finding the optimal projection $g$ of an integrand $f$ onto certain classes of piecewise constant functions, in order to minimize the variance of the unbiased importance sampling estimator $E_g[f/g]$, as well as the related problem of finding optimal mixture weights to approximate and importance sample a target mixture distribution $f = \sum_i α_i f_i$ with components $f_i$ in a family $\mathcal{F}$, through a corresponding mixture of importance sampling densities $g_i$ that are only approximately proportional to $f_i$. We further show that in both cases the optimal projection is different from the commonly used $\ell_1$ projection, and provide an intuitive explanation for the difference.