The Shooting Regressor; Randomized Gradient-Based Ensembles
This is an incremental improvement for machine learning practitioners seeking better ensemble methods.
The paper introduces an ensemble method that uses randomization and loss function gradients to make predictions, with empirical results showing it can outperform existing techniques in accuracy on a popular dataset.
An ensemble method is introduced that utilizes randomization and loss function gradients to compute a prediction. Multiple weakly-correlated estimators approximate the gradient at randomly sampled points on the error surface and are aggregated into a final solution. A scaling parameter is described that controls a trade-off between ensemble correlation and precision. Numerical methods for estimating optimal values of the parameter are described. Empirical results are computed over a popular dataset. Inferential statistics on these results show that the method is capable of outperforming existing techniques in terms of increased accuracy.