Contrastive Neural Ratio Estimation for Simulation-based Inference
This work addresses a bias problem in simulation-based inference for practitioners, offering an incremental improvement over existing methods.
The authors tackled the bias issue in multiclass likelihood-to-evidence ratio estimation (NRE-B) by proposing a new multiclass framework that eliminates this bias, enabling reliable diagnostics and recovering existing methods in specific cases. They benchmarked algorithms under various data regimes, revealing optimal hyperparameters distinct from prior models and suggesting mutual information bounds as performance metrics.
Likelihood-to-evidence ratio estimation is usually cast as either a binary (NRE-A) or a multiclass (NRE-B) classification task. In contrast to the binary classification framework, the current formulation of the multiclass version has an intrinsic and unknown bias term, making otherwise informative diagnostics unreliable. We propose a multiclass framework free from the bias inherent to NRE-B at optimum, leaving us in the position to run diagnostics that practitioners depend on. It also recovers NRE-A in one corner case and NRE-B in the limiting case. For fair comparison, we benchmark the behavior of all algorithms in both familiar and novel training regimes: when jointly drawn data is unlimited, when data is fixed but prior draws are unlimited, and in the commonplace fixed data and parameters setting. Our investigations reveal that the highest performing models are distant from the competitors (NRE-A, NRE-B) in hyperparameter space. We make a recommendation for hyperparameters distinct from the previous models. We suggest two bounds on the mutual information as performance metrics for simulation-based inference methods, without the need for posterior samples, and provide experimental results. This version corrects a minor implementation error in $γ$, improving results.