Multi-Modal Mean-Fields via Cardinality-Based Clamping
This work addresses a limitation in inference methods for computer vision and statistical physics, but it appears incremental as it builds on existing clamping methods and diverse MAP paradigms.
The paper tackled the problem of Mean Field inference failing to capture dependencies between variables in Conditional Random Fields by proposing a weighted mixture of factorized distributions that minimizes KL-Divergence to the true posterior. The result showed positive impacts on real-world algorithms that use mean fields, though no concrete numbers were provided.
Mean Field inference is central to statistical physics. It has attracted much interest in the Computer Vision community to efficiently solve problems expressible in terms of large Conditional Random Fields. However, since it models the posterior probability distribution as a product of marginal probabilities, it may fail to properly account for important dependencies between variables. We therefore replace the fully factorized distribution of Mean Field by a weighted mixture of such distributions, that similarly minimizes the KL-Divergence to the true posterior. By introducing two new ideas, namely, conditioning on groups of variables instead of single ones and using a parameter of the conditional random field potentials, that we identify to the temperature in the sense of statistical physics to select such groups, we can perform this minimization efficiently. Our extension of the clamping method proposed in previous works allows us to both produce a more descriptive approximation of the true posterior and, inspired by the diverse MAP paradigms, fit a mixture of Mean Field approximations. We demonstrate that this positively impacts real-world algorithms that initially relied on mean fields.