The Limits of Tractable Marginalization
This addresses a foundational question in computational complexity and probabilistic inference, showing limitations in representing tractable marginalization, which is incremental but clarifies theoretical boundaries.
The paper tackles the problem of whether all functions with polynomial-time marginalization algorithms can be succinctly expressed by polynomial-size arithmetic circuits, and it gives a negative answer by exhibiting simple functions with tractable marginalization but no efficient representation under standard complexity assumptions.
Marginalization -- summing a function over all assignments to a subset of its inputs -- is a fundamental computational problem with applications from probabilistic inference to formal verification. Despite its computational hardness in general, there exist many classes of functions (e.g., probabilistic models) for which marginalization remains tractable, and they can be commonly expressed by polynomial size arithmetic circuits computing multilinear polynomials. This raises the question, can all functions with polynomial time marginalization algorithms be succinctly expressed by such circuits? We give a negative answer, exhibiting simple functions with tractable marginalization yet no efficient representation by known models, assuming $\textsf{FP}\neq\#\textsf{P}$ (an assumption implied by $\textsf{P} \neq \textsf{NP}$). To this end, we identify a hierarchy of complexity classes corresponding to stronger forms of marginalization, all of which are efficiently computable on the known circuit models. We conclude with a completeness result, showing that whenever there is an efficient real RAM performing virtual evidence marginalization for a function, then there are small circuits for that function's multilinear representation.