Analysis of Bayesian Inference Algorithms by the Dynamical Functional Approach
This work provides theoretical insights into algorithmic performance for statistical inference in complex models, but it is incremental as it builds on existing methods like the replica method and dynamical functional approach.
The paper tackled the analysis of convergence rates for Bayesian inference algorithms in large Gaussian latent variable models with random covariance matrices, achieving exact closed-form expressions for convergence rates that show excellent agreement with simulations.
We analyze the dynamics of an algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. For the case of perfect data-model matching, the knowledge of static order parameters derived from the replica method allows us to obtain efficient algorithmic updates in terms of matrix-vector multiplications with a fixed matrix. Using the dynamical functional approach, we obtain an exact effective stochastic process in the thermodynamic limit for a single node. From this, we obtain closed-form expressions for the rate of the convergence. Analytical results are excellent agreement with simulations of single instances of large models.