Sufficiency, Separability and Temporal Probabilistic Models
This work addresses a foundational problem in probabilistic modeling for researchers, offering a theoretical framework to improve inference efficiency in dynamic systems, but it is incremental as it builds on existing concepts of separability.
The paper investigates when marginal distributions of conditioning variables suffice to determine a conditional probability, linking this to additive separability and its generalization to conditional separability. It shows that in temporal probabilistic models, separability enables tractable exact prediction by propagating marginal subsystem probabilities, though observations can break this separability, making exact monitoring difficult.
Suppose we are given the conditional probability of one variable given some other variables.Normally the full joint distribution over the conditioning variablesis required to determine the probability of the conditioned variable.Under what circumstances are the marginal distributions over the conditioning variables sufficient to determine the probability ofthe conditioned variable?Sufficiency in this sense is equivalent to additive separability ofthe conditional probability distribution.Such separability structure is natural and can be exploited forefficient inference.Separability has a natural generalization to conditional separability.Separability provides a precise notion of weaklyinteracting subsystems in temporal probabilistic models.Given a system that is decomposed into separable subsystems, exactmarginal probabilities over subsystems at future points in time can becomputed by propagating marginal subsystem probabilities, rather thancomplete system joint probabilities.Thus, separability can make exact prediction tractable.However, observations can break separability,so exact monitoring of dynamic systems remains hard.