Per-sample Prediction Intervals for Extreme Learning Machines
This addresses the need for reliable uncertainty quantification in applied machine learning, particularly for heteroscedastic problems, but is incremental as it builds on existing Extreme Learning Machine and Jackknife techniques.
The paper tackles the problem of estimating input-dependent prediction intervals for heteroscedastic outputs in supervised machine learning by proposing a method using an Extreme Learning Machine with a weighted Jackknife variance correction, resulting in a fast and robust approach that handles large datasets and limited training data.
Prediction intervals in supervised Machine Learning bound the region where the true outputs of new samples may fall. They are necessary in the task of separating reliable predictions of a trained model from near random guesses, minimizing the rate of False Positives, and other problem-specific tasks in applied Machine Learning. Many real problems have heteroscedastic stochastic outputs, which explains the need of input-dependent prediction intervals. This paper proposes to estimate the input-dependent prediction intervals by a separate Extreme Learning Machine model, using variance of its predictions as a correction term accounting for the model uncertainty. The variance is estimated from the model's linear output layer with a weighted Jackknife method. The methodology is very fast, robust to heteroscedastic outputs, and handles both extremely large datasets and insufficient amount of training data.