Sum-Product-Quotient Networks
This work addresses the expressivity gap for researchers in probabilistic modeling, offering a novel extension to SPNs that is incremental but provides theoretical efficiency gains.
The paper tackles the limited expressivity of tractable generative models like Sum-Product Networks (SPNs) by introducing Sum-Product-Quotient Networks (SPQNs), which incorporate conditional distributions via quotient nodes, resulting in an exponential boost in expressive efficiency proven by distributions that SPQNs compute efficiently but require SPNs to be of exponential size.
We present a novel tractable generative model that extends Sum-Product Networks (SPNs) and significantly boosts their power. We call it Sum-Product-Quotient Networks (SPQNs), whose core concept is to incorporate conditional distributions into the model by direct computation using quotient nodes, e.g. $P(A|B) = \frac{P(A,B)}{P(B)}$. We provide sufficient conditions for the tractability of SPQNs that generalize and relax the decomposable and complete tractability conditions of SPNs. These relaxed conditions give rise to an exponential boost to the expressive efficiency of our model, i.e. we prove that there are distributions which SPQNs can compute efficiently but require SPNs to be of exponential size. Thus, we narrow the gap in expressivity between tractable graphical models and other Neural Network-based generative models.