Bootstrap Bias Corrected Cross Validation applied to Super Learning
This work addresses verification efficiency for researchers using super learners, but it is incremental as it adapts an existing resampling idea to a specific context.
The paper tackled the problem of verifying super learner predictions by proposing Bootstrap Bias Correction as an alternative to nested cross-validation, showing it to be reasonably precise and very cost-efficient in tests on artificial and biomedical datasets.
Super learner algorithm can be applied to combine results of multiple base learners to improve quality of predictions. The default method for verification of super learner results is by nested cross validation. It has been proposed by Tsamardinos et al., that nested cross validation can be replaced by resampling for tuning hyper-parameters of the learning algorithms. We apply this idea to verification of super learner and compare with other verification methods, including nested cross validation. Tests were performed on artificial data sets of diverse size and on seven real, biomedical data sets. The resampling method, called Bootstrap Bias Correction, proved to be a reasonably precise and very cost-efficient alternative for nested cross validation.