Loss-calibrated expectation propagation for approximate Bayesian decision-making
This work addresses the challenge of making better decisions in machine learning by enhancing approximate Bayesian methods to account for utility functions, though it is incremental as it builds on existing expectation propagation techniques.
The paper tackled the problem of approximate Bayesian inference for decision-making by introducing Loss-EP, a variant of expectation propagation that incorporates utility functions to improve decision sensitivity, and demonstrated its application in Gaussian process classification with asymmetric error penalties, showing significant impacts on approximation usefulness.
Approximate Bayesian inference methods provide a powerful suite of tools for finding approximations to intractable posterior distributions. However, machine learning applications typically involve selecting actions, which -- in a Bayesian setting -- depend on the posterior distribution only via its contribution to expected utility. A growing body of work on loss-calibrated approximate inference methods has therefore sought to develop posterior approximations sensitive to the influence of the utility function. Here we introduce loss-calibrated expectation propagation (Loss-EP), a loss-calibrated variant of expectation propagation. This method resembles standard EP with an additional factor that "tilts" the posterior towards higher-utility decisions. We show applications to Gaussian process classification under binary utility functions with asymmetric penalties on False Negative and False Positive errors, and show how this asymmetry can have dramatic consequences on what information is "useful" to capture in an approximation.