Expectation Propagation for t-Exponential Family Using Q-Algebra
This work addresses a specific bottleneck in probabilistic modeling for noisy data, offering an incremental improvement by extending EP to a new distribution family.
The paper tackled the problem of efficiently learning t-exponential family distributions, which handle noisy data but lack algorithms like expectation propagation, by developing an EP algorithm using q-algebra, and demonstrated its performance in applications like Bayes point machine and Student-t process classification.
Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters. The exponential family has recently been extended to the t-exponential family, which contains Student-t distributions as family members and thus allows us to handle noisy data well. However, since the t-exponential family is denied by the deformed exponential, we cannot derive an efficient learning algorithm for the t-exponential family such as expectation propagation (EP). In this paper, we borrow the mathematical tools of q-algebra from statistical physics and show that the pseudo additivity of distributions allows us to perform calculation of t-exponential family distributions through natural parameters. We then develop an expectation propagation (EP) algorithm for the t-exponential family, which provides a deterministic approximation to the posterior or predictive distribution with simple moment matching. We finally apply the proposed EP algorithm to the Bayes point machine and Student-t process classication, and demonstrate their performance numerically.