Inversion of Bayesian Networks
This work addresses a foundational theoretical problem in probabilistic graphical modeling, with potential implications for improving variational inference methods, but it is incremental as it builds on existing frameworks like variational autoencoders.
The paper tackles the problem of determining the necessary and sufficient properties for a recognition network to exactly model the true posterior distribution in Bayesian networks, deriving both global and local conditions based on d-separation and perfectness.
Variational autoencoders and Helmholtz machines use a recognition network (encoder) to approximate the posterior distribution of a generative model (decoder). In this paper we study the necessary and sufficient properties of a recognition network so that it can model the true posterior distribution exactly. These results are derived in the general context of probabilistic graphical modelling / Bayesian networks, for which the network represents a set of conditional independence statements. We derive both global conditions, in terms of d-separation, and local conditions for the recognition network to have the desired qualities. It turns out that for the local conditions the property perfectness (for every node, all parents are joined) plays an important role.