Learning Approximately Objective Priors
This addresses the problem of intractable prior specification in Bayesian modeling, offering a practical solution for modelers, though it is incremental as it builds on existing objective prior frameworks.
The paper tackles the challenge of deriving objective priors like Jeffreys and reference priors for complex models by proposing a method to learn approximations through parametric families and optimization, demonstrating effectiveness by recovering Jeffreys priors and learning a reference prior for Variational Autoencoders.
Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques for learning reference prior approximations: we select a parametric family and optimize a black-box lower bound on the reference prior objective to find the member of the family that serves as a good approximation. We experimentally demonstrate the method's effectiveness by recovering Jeffreys priors and learning the Variational Autoencoder's reference prior.