Multi-class SVMs: From Tighter Data-Dependent Generalization Bounds to Novel Algorithms
This work addresses generalization performance in multi-class classification, offering improved theoretical bounds and practical algorithms for machine learning applications.
The paper tackles the problem of multi-class classification by deriving a data-dependent generalization error bound with logarithmic dependence on class size, improving upon previous linear bounds, and introduces a new algorithm based on $\ell_p$-norm regularization that achieves significant accuracy gains on real-world datasets.
This paper studies the generalization performance of multi-class classification algorithms, for which we obtain, for the first time, a data-dependent generalization error bound with a logarithmic dependence on the class size, substantially improving the state-of-the-art linear dependence in the existing data-dependent generalization analysis. The theoretical analysis motivates us to introduce a new multi-class classification machine based on $\ell_p$-norm regularization, where the parameter $p$ controls the complexity of the corresponding bounds. We derive an efficient optimization algorithm based on Fenchel duality theory. Benchmarks on several real-world datasets show that the proposed algorithm can achieve significant accuracy gains over the state of the art.