Towards Robust Classification with Deep Generative Forests
This work addresses the lack of principled uncertainty methods in widely used discriminative models like random forests, offering a solution for applications requiring robust classification.
The paper tackles the problem of uncertainty quantification in decision trees and random forests by introducing Generative Forests (GeFs), which extend these models to represent the full joint distribution over features, resulting in uncertainty-aware classifiers capable of measuring prediction robustness and detecting out-of-distribution samples.
Decision Trees and Random Forests are among the most widely used machine learning models, and often achieve state-of-the-art performance in tabular, domain-agnostic datasets. Nonetheless, being primarily discriminative models they lack principled methods to manipulate the uncertainty of predictions. In this paper, we exploit Generative Forests (GeFs), a recent class of deep probabilistic models that addresses these issues by extending Random Forests to generative models representing the full joint distribution over the feature space. We demonstrate that GeFs are uncertainty-aware classifiers, capable of measuring the robustness of each prediction as well as detecting out-of-distribution samples.