An Infra-Structure for Performance Estimation and Experimental Comparison of Predictive Models in R
This work addresses the need for a standardized tool to compare predictive models in R, but it is incremental as it builds on existing performance estimation concepts without introducing new methods.
The authors tackled the problem of estimating predictive performance for various modeling approaches in R by developing the performanceEstimation package, which provides a generic infrastructure for classification, regression, and time series tasks, enabling users to easily set up experiments with minimal effort.
This document describes an infra-structure provided by the R package performanceEstimation that allows to estimate the predictive performance of different approaches (workflows) to predictive tasks. The infra-structure is generic in the sense that it can be used to estimate the values of any performance metrics, for any workflow on different predictive tasks, namely, classification, regression and time series tasks. The package also includes several standard workflows that allow users to easily set up their experiments limiting the amount of work and information they need to provide. The overall goal of the infra-structure provided by our package is to facilitate the task of estimating the predictive performance of different modeling approaches to predictive tasks in the R environment.