Iterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk
This provides a method for statisticians and machine learning practitioners to compute reference priors and minimax risk, with broader applicability to optimization problems, but it is incremental as it adapts existing techniques to new contexts.
The paper tackles the computation of reference priors and minimax risk for parametric families by developing an iterative Markov chain Monte Carlo algorithm based on the Blahut-Arimoto algorithm, with a statistical analysis bounding the required samples for approximation and simulations in exponential families.
We present an iterative Markov chainMonte Carlo algorithm for computingreference priors and minimax risk forgeneral parametric families. Ourapproach uses MCMC techniques based onthe Blahut-Arimoto algorithm forcomputing channel capacity ininformation theory. We give astatistical analysis of the algorithm,bounding the number of samples requiredfor the stochastic algorithm to closelyapproximate the deterministic algorithmin each iteration. Simulations arepresented for several examples fromexponential families. Although we focuson applications to reference priors andminimax risk, the methods and analysiswe develop are applicable to a muchbroader class of optimization problemsand iterative algorithms.