BR-SNIS: Bias Reduced Self-Normalized Importance Sampling
This work addresses bias reduction in importance sampling for practitioners in statistics and machine learning, though it is incremental as it builds on existing SNIS with a wrapper approach.
The authors tackled the bias problem in self-normalized importance sampling (SNIS) by proposing BR-SNIS, a method that reduces bias without increasing variance, achieving significant bias reduction as shown in theoretical bounds and numerical examples.
Importance Sampling (IS) is a method for approximating expectations under a target distribution using independent samples from a proposal distribution and the associated importance weights. In many applications, the target distribution is known only up to a normalization constant, in which case self-normalized IS (SNIS) can be used. While the use of self-normalization can have a positive effect on the dispersion of the estimator, it introduces bias. In this work, we propose a new method, BR-SNIS, whose complexity is essentially the same as that of SNIS and which significantly reduces bias without increasing the variance. This method is a wrapper in the sense that it uses the same proposal samples and importance weights as SNIS, but makes clever use of iterated sampling--importance resampling (ISIR) to form a bias-reduced version of the estimator. We furnish the proposed algorithm with rigorous theoretical results, including new bias, variance and high-probability bounds, and these are illustrated by numerical examples.