Conditional Risk Minimization for Stochastic Processes
This addresses the problem of improving prediction accuracy for sequential data in machine learning, though it appears incremental as it builds on existing non-parametric time-series theory.
The paper tackles learning from non-i.i.d. data by developing predictors that minimize conditional risk for stochastic processes, resulting in a practical estimator with a finite sample concentration bound for uniform convergence under regularity assumptions.
We study the task of learning from non-i.i.d. data. In particular, we aim at learning predictors that minimize the conditional risk for a stochastic process, i.e. the expected loss of the predictor on the next point conditioned on the set of training samples observed so far. For non-i.i.d. data, the training set contains information about the upcoming samples, so learning with respect to the conditional distribution can be expected to yield better predictors than one obtains from the classical setting of minimizing the marginal risk. Our main contribution is a practical estimator for the conditional risk based on the theory of non-parametric time-series prediction, and a finite sample concentration bound that establishes uniform convergence of the estimator to the true conditional risk under certain regularity assumptions on the process.