Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference
This addresses simulation-based inference challenges in scientific domains like physics, offering a method for source distribution estimation when likelihoods are intractable, though it appears incremental as it builds on existing neural density estimation techniques.
The paper tackles the problem of empirical Bayes estimation without a tractable likelihood, using neural density estimators to estimate a source distribution from noise-corrupted observations and perform posterior inference. It shows that the approach recovers ground truth source distributions up to symmetries and demonstrates applicability in likelihood-free inference and a collider physics inverse problem.
We revisit empirical Bayes in the absence of a tractable likelihood function, as is typical in scientific domains relying on computer simulations. We investigate how the empirical Bayesian can make use of neural density estimators first to use all noise-corrupted observations to estimate a prior or source distribution over uncorrupted samples, and then to perform single-observation posterior inference using the fitted source distribution. We propose an approach based on the direct maximization of the log-marginal likelihood of the observations, examining both biased and de-biased estimators, and comparing to variational approaches. We find that, up to symmetries, a neural empirical Bayes approach recovers ground truth source distributions. With the learned source distribution in hand, we show the applicability to likelihood-free inference and examine the quality of the resulting posterior estimates. Finally, we demonstrate the applicability of Neural Empirical Bayes on an inverse problem from collider physics.