Total Stability of SVMs and Localized SVMs
This work addresses the robustness of SVMs for practitioners dealing with data variations, though it is incremental as it builds on prior stability research.
The paper investigates the stability of support vector machines (SVMs) and localized SVMs under simultaneous small perturbations in the probability measure, regularization parameter, and kernel, generalizing and improving existing results, and extends these findings to localized learning for big data applications.
Regularized kernel-based methods such as support vector machines (SVMs) typically depend on the underlying probability measure $\mathrm{P}$ (respectively an empirical measure $\mathrm{D}_n$ in applications) as well as on the regularization parameter $λ$ and the kernel $k$. Whereas classical statistical robustness only considers the effect of small perturbations in $\mathrm{P}$, the present paper investigates the influence of simultaneous slight variations in the whole triple $(\mathrm{P},λ,k)$, respectively $(\mathrm{D}_n,λ_n,k)$, on the resulting predictor. Existing results from the literature are considerably generalized and improved. In order to also make them applicable to big data, where regular SVMs suffer from their super-linear computational requirements, we show how our results can be transferred to the context of localized learning. Here, the effect of slight variations in the applied regionalization, which might for example stem from changes in $\mathrm{P}$ respectively $\mathrm{D}_n$, is considered as well.