Asymptotic Confidence Sets for General Nonparametric Regression and Classification by Regularized Kernel Methods
This work provides statistical inference tools for kernel methods, filling a gap in mathematical statistics for researchers and practitioners using these algorithms.
The paper addresses the lack of statistical inference results for regularized kernel methods like support vector machines, deriving a strongly consistent estimator for the covariance matrix to obtain asymptotically correct confidence sets for functionals of the minimizer, enabling applications such as pointwise confidence sets for values, gradients, integrals, and norms.
Regularized kernel methods such as, e.g., support vector machines and least-squares support vector regression constitute an important class of standard learning algorithms in machine learning. Theoretical investigations concerning asymptotic properties have manly focused on rates of convergence during the last years but there are only very few and limited (asymptotic) results on statistical inference so far. As this is a serious limitation for their use in mathematical statistics, the goal of the article is to fill this gap. Based on asymptotic normality of many of these methods, the article derives a strongly consistent estimator for the unknown covariance matrix of the limiting normal distribution. In this way, we obtain asymptotically correct confidence sets for $ψ(f_{P,λ_0})$ where $f_{P,λ_0}$ denotes the minimizer of the regularized risk in the reproducing kernel Hilbert space $H$ and $ψ:H\rightarrow\mathds{R}^m$ is any Hadamard-differentiable functional. Applications include (multivariate) pointwise confidence sets for values of $f_{P,λ_0}$ and confidence sets for gradients, integrals, and norms.