Expectation Propagation for Approximate Inference: Free Probability Framework
This work addresses scalability issues in approximate inference for machine learning practitioners, though it is incremental as it builds on existing EP methods with specific data assumptions.
The authors tackled the computational bottleneck of expectation propagation (EP) in large-scale approximate inference by using free probability theory to analyze asymptotic properties, enabling efficient application to gene selection in microarray datasets.
We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate Gaussian approximation to the exact posterior distribution. The computational complexity of the repeated update of covariance matrices severely limits the application of EP to large problem sizes. In this study, we present a rigorous analysis by means of free probability theory that allows us to overcome this computational bottleneck if specific data matrices in the problem fulfill certain properties of asymptotic freeness. We demonstrate the relevance of our approach on the gene selection problem of a microarray dataset.