Structured Message Passing
This work addresses the challenge of computational inefficiency in probabilistic inference for machine learning and AI, offering a novel framework that unifies and extends existing methods, though it is incremental in building on prior algorithms.
The paper tackles the problem of improving approximate inference algorithms by introducing structured message passing (SMP), a framework that exploits structured representations like algebraic decision diagrams to enhance efficiency and accuracy, resulting in new algorithms that are shown to be more accurate and scalable than state-of-the-art techniques.
In this paper, we present structured message passing (SMP), a unifying framework for approximate inference algorithms that take advantage of structured representations such as algebraic decision diagrams and sparse hash tables. These representations can yield significant time and space savings over the conventional tabular representation when the message has several identical values (context-specific independence) or zeros (determinism) or both in its range. Therefore, in order to fully exploit the power of structured representations, we propose to artificially introduce context-specific independence and determinism in the messages. This yields a new class of powerful approximate inference algorithms which includes popular algorithms such as cluster-graph Belief propagation (BP), expectation propagation and particle BP as special cases. We show that our new algorithms introduce several interesting bias-variance trade-offs. We evaluate these trade-offs empirically and demonstrate that our new algorithms are more accurate and scalable than state-of-the-art techniques.