metabeta -- A fast neural model for Bayesian mixed-effects regression
This addresses the problem of slow Bayesian inference for researchers in empirical sciences dealing with hierarchical data, offering a faster alternative to MCMC.
The paper tackles the computational cost of Bayesian mixed-effects regression by introducing metabeta, a transformer-based neural model for neural posterior estimation, achieving comparable performance to MCMC methods in significantly less time.
Hierarchical data with multiple observations per group is ubiquitous in empirical sciences and is often analyzed using mixed-effects regression. In such models, Bayesian inference gives an estimate of uncertainty but is analytically intractable and requires costly approximation using Markov Chain Monte Carlo (MCMC) methods. Neural posterior estimation shifts the bulk of computation from inference time to pre-training time, amortizing over simulated datasets with known ground truth targets. We propose metabeta, a transformer-based neural network model for Bayesian mixed-effects regression. Using simulated and real data, we show that it reaches stable and comparable performance to MCMC-based parameter estimation at a fraction of the usually required time.