RRMSE Voting Regressor: A weighting function based improvement to ensemble regression
This work addresses the challenge of assigning effective weights in ensemble regression without domain knowledge, offering an incremental improvement for machine learning practitioners in regression tasks.
The paper tackled the problem of improving ensemble regression predictions by proposing an RRMSE-based weighting function to assign non-uniform weights to base models, resulting in significantly better performance than other state-of-the-art ensemble methods on six datasets.
This paper describes the RRMSE (Relative Root Mean Square Error) based weights to weight the occurrences of predictive values before averaging for the ensemble voting regression. The core idea behind ensemble regression is to combine several base regression models in order to improve the prediction performance in learning problems with a numeric continuous target variable. The default weights setting for the ensemble voting regression is uniform weights, and without domain knowledge of learning task, assigning weights for predictions are impossible, which makes it very difficult to improve the predictions. This work attempts to improve the prediction of voting regression by implementing the RRMSE based weighting function. Experiments show that RRMSE voting regressor produces significantly better predictions than other state-of-the-art ensemble regression algorithms on six popular regression learning datasets.