Analysis of Regularized Least Squares in Reproducing Kernel Krein Spaces
This work provides the first approximation analysis for regularized learning algorithms in RKKS, addressing a theoretical gap for researchers in kernel methods and non-convex optimization.
The paper tackles the theoretical analysis of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS), demonstrating that the problem has a globally optimal solution with a closed form under a bounded constraint and deriving learning rates that match those in reproducing kernel Hilbert spaces (RKHS) under certain conditions.
In this paper, we study the asymptotic properties of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS). By introducing a bounded hyper-sphere constraint to such non-convex regularized risk minimization problem, we theoretically demonstrate that this problem has a globally optimal solution with a closed form on the sphere, which makes approximation analysis feasible in RKKS. Regarding to the original regularizer induced by the indefinite inner product, we modify traditional error decomposition techniques, prove convergence results for the introduced hypothesis error based on matrix perturbation theory, and derive learning rates of such regularized regression problem in RKKS. Under some conditions, the derived learning rates in RKKS are the same as that in reproducing kernel Hilbert spaces (RKHS), which is actually the first work on approximation analysis of regularized learning algorithms in RKKS.