Local Support Vector Machines:Formulation and Analysis
This work addresses the need for localized, nonparametric classification methods in machine learning, representing an incremental advancement in SVM theory.
The paper tackles the problem of developing Local Support Vector Machines (LSVMs) by formulating a generalization of previous methods and analyzing Local Linear SVMs (LLSVMs), establishing conditions for Bayes consistency and convergence rates for local risk, with generalization error bounds derived using stability arguments.
We provide a formulation for Local Support Vector Machines (LSVMs) that generalizes previous formulations, and brings out the explicit connections to local polynomial learning used in nonparametric estimation literature. We investigate the simplest type of LSVMs called Local Linear Support Vector Machines (LLSVMs). For the first time we establish conditions under which LLSVMs make Bayes consistent predictions at each test point $x_0$. We also establish rates at which the local risk of LLSVMs converges to the minimum value of expected local risk at each point $x_0$. Using stability arguments we establish generalization error bounds for LLSVMs.