Martingale Posterior Neural Networks for Fast Sequential Decision Making
This addresses the need for scalable and fast uncertainty quantification in online decision-making problems, offering a significant speed improvement over existing methods.
The paper tackles the problem of slow Bayesian sequential decision making by introducing a predictive-first approach using martingale posteriors, which achieves 10-100 times faster inference than classical Thompson sampling while maintaining competitive or superior performance in tasks like non-stationary contextual bandits and Bayesian optimization.
We introduce scalable algorithms for online learning of neural network parameters and Bayesian sequential decision making. Unlike classical Bayesian neural networks, which induce predictive uncertainty through a posterior over model parameters, our methods adopt a predictive-first perspective based on martingale posteriors. In particular, we work directly with the one-step-ahead posterior predictive, which we parameterize with a neural network and update sequentially with incoming observations. This decouples Bayesian decision-making from parameter-space inference: we sample from the posterior predictive for decision making, and update the parameters of the posterior predictive via fast, frequentist Kalman-filter-like recursions. Our algorithms operate in a fully online, replay-free setting, providing principled uncertainty quantification without costly posterior sampling. Empirically, they achieve competitive performance-speed trade-offs in non-stationary contextual bandits and Bayesian optimization, offering 10-100 times faster inference than classical Thompson sampling while maintaining comparable or superior decision performance.