Sum-Product Networks for Hybrid Domains
This addresses the problem of time-consuming model specification for users in fields like data science and machine learning dealing with mixed data, representing a novel method for a known bottleneck.
The paper tackles the difficulty of building probabilistic graphical models for hybrid domains with mixed data types by proposing Mixed SPNs, a trainable probabilistic deep architecture that eliminates the need for users to pre-specify parametric forms of random variables, achieving effective approximation of continuous distributions and efficient learning and inference.
While all kinds of mixed data -from personal data, over panel and scientific data, to public and commercial data- are collected and stored, building probabilistic graphical models for these hybrid domains becomes more difficult. Users spend significant amounts of time in identifying the parametric form of the random variables (Gaussian, Poisson, Logit, etc.) involved and learning the mixed models. To make this difficult task easier, we propose the first trainable probabilistic deep architecture for hybrid domains that features tractable queries. It is based on Sum-Product Networks (SPNs) with piecewise polynomial leave distributions together with novel nonparametric decomposition and conditioning steps using the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. This relieves the user from deciding a-priori the parametric form of the random variables but is still expressive enough to effectively approximate any continuous distribution and permits efficient learning and inference. Our empirical evidence shows that the architecture, called Mixed SPNs, can indeed capture complex distributions across a wide range of hybrid domains.