Partition Trees: Conditional Density Estimation over General Outcome Spaces
This provides a scalable, nonparametric alternative for conditional density estimation that works with both continuous and categorical variables, addressing a need in probabilistic modeling.
The authors tackled conditional density estimation over general outcome spaces by proposing Partition Trees, a tree-based framework that models conditional distributions as piecewise-constant densities on data-adaptive partitions. They demonstrated improved probabilistic prediction over CART-style trees and competitive or superior performance compared to state-of-the-art probabilistic tree methods and Random Forests.
We propose Partition Trees, a tree-based framework for conditional density estimation over general outcome spaces, supporting both continuous and categorical variables within a unified formulation. Our approach models conditional distributions as piecewise-constant densities on data adaptive partitions and learns trees by directly minimizing conditional negative log-likelihood. This yields a scalable, nonparametric alternative to existing probabilistic trees that does not make parametric assumptions about the target distribution. We further introduce Partition Forests, an ensemble extension obtained by averaging conditional densities. Empirically, we demonstrate improved probabilistic prediction over CART-style trees and competitive or superior performance compared to state-of-the-art probabilistic tree methods and Random Forests, along with robustness to redundant features and heteroscedastic noise.