Analysis of Random Sequential Message Passing Algorithms for Approximate Inference
This work addresses convergence issues in approximate inference algorithms for complex statistical models, but it is incremental as it builds on existing message passing and stability analysis frameworks.
The authors analyzed the dynamics of a random sequential message passing algorithm for approximate inference in Gaussian latent variable models, deriving exact mean-field equations and identifying parameter ranges where the algorithm fails to converge, linking this to the de Almeida Thouless stability condition.
We analyze the dynamics of a random sequential message passing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one used by the student may be different. By means of dynamical functional approach, we obtain exact dynamical mean-field equations characterizing the dynamics of the inference algorithm. We also derive a range of model parameters for which the sequential algorithm does not converge. The boundary of this parameter range coincides with the de Almeida Thouless (AT) stability condition of the replica symmetric ansatz for the static probabilistic model.