Lp- and Risk Consistency of Localized SVMs
This addresses scalability issues for practitioners using SVMs in large-scale machine learning applications, though it is incremental as it builds on existing SVM theory.
The paper tackles the computational inefficiency of support vector machines (SVMs) on large datasets by analyzing localized SVMs, proving that they inherit Lp- and risk consistency from global SVMs under weak conditions, even with changing regions as data size increases.
Kernel-based regularized risk minimizers, also called support vector machines (SVMs), are known to possess many desirable properties but suffer from their super-linear computational requirements when dealing with large data sets. This problem can be tackled by using localized SVMs instead, which also offer the additional advantage of being able to apply different hyperparameters to different regions of the input space. In this paper, localized SVMs are analyzed with regards to their consistency. It is proven that they inherit $L_p$- as well as risk consistency from global SVMs under very weak conditions and even if the regions underlying the localized SVMs are allowed to change as the size of the training data set increases.