Nonparametric Probabilistic Regression with Coarse Learners
This method addresses the problem of accurate density estimation in regression for applications requiring uncertainty quantification, though it appears incremental as it builds on existing classifier techniques.
The paper tackles probabilistic regression by predicting full conditional density functions using a nonparametric method that combines base classifiers trained on coarsened target values, achieving competitive performance with high-fidelity prediction intervals on various datasets.
Probabilistic Regression refers to predicting a full probability density function for the target conditional on the features. We present a nonparametric approach to this problem which combines base classifiers (typically gradient boosted forests) trained on different coarsenings of the target value. By combining such classifiers and averaging the resulting densities, we are able to compute precise conditional densities with minimal assumptions on the shape or form of the density. We combine this approach with a structured cross-entropy loss function which serves to regularize and smooth the resulting densities. Prediction intervals computed from these densities are shown to have high fidelity in practice. Furthermore, examining the properties of these densities on particular observations can provide valuable insight. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.