Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference
This addresses the problem of efficient, parallelizable inference for model comparison and uncertainty quantification in fields like Bayesian statistics and machine learning, representing a novel method for a known bottleneck rather than a foundational breakthrough.
The paper tackles the challenge of scalable inference for complex, multimodal models by introducing Nested Slice Sampling (NSS), a GPU-friendly, vectorized version of Nested Sampling that maintains accurate evidence estimates and high-quality posterior samples, showing robustness on difficult multimodal problems where state-of-the-art methods like tempered SMC struggle.
Model comparison and calibrated uncertainty quantification often require integrating over parameters, but scalable inference can be challenging for complex, multimodal targets. Nested Sampling is a robust alternative to standard MCMC, yet its typically sequential structure and hard constraints make efficient accelerator implementations difficult. This paper introduces Nested Slice Sampling (NSS), a GPU-friendly, vectorized formulation of Nested Sampling that uses Hit-and-Run Slice Sampling for constrained updates. A tuning analysis yields a simple near-optimal rule for setting the slice width, improving high-dimensional behavior and making per-step compute more predictable for parallel execution. Experiments on challenging synthetic targets, high dimensional Bayesian inference, and Gaussian process hyperparameter marginalization show that NSS maintains accurate evidence estimates and high-quality posterior samples, and is particularly robust on difficult multimodal problems where current state-of-the-art methods such as tempered SMC baselines can struggle. An open-source implementation is released to facilitate adoption and reproducibility.