Learning Arbitrary Sum-Product Network Leaves with Expectation-Maximization
This work addresses a specific bottleneck in probabilistic modeling for researchers and practitioners, offering an incremental improvement by extending learning capabilities to complex leaf distributions in SPNs.
The paper tackles the problem of learning complex leaf distributions in Sum-Product Networks, which was previously an open issue with limited solutions, by deriving an efficient Expectation-Maximization method that allows learning a broad class of distributions with linear cost and convergence even under partial optimization. The result shows that their model outperforms state-of-the-art methods in density estimation on twenty real-life datasets while using fewer parameters.
Sum-Product Networks with complex probability distribution at the leaves have been shown to be powerful tractable-inference probabilistic models. However, while learning the internal parameters has been amply studied, learning complex leaf distribution is an open problem with only few results available in special cases. In this paper we derive an efficient method to learn a very large class of leaf distributions with Expectation-Maximization. The EM updates have the form of simple weighted maximum likelihood problems, allowing to use any distribution that can be learned with maximum likelihood, even approximately. The algorithm has cost linear in the model size and converges even if only partial optimizations are performed. We demonstrate this approach with experiments on twenty real-life datasets for density estimation, using tree graphical models as leaves. Our model outperforms state-of-the-art methods for parameter learning despite using SPNs with much fewer parameters.