Random matrices meet machine learning: a large dimensional analysis of LS-SVM
This provides theoretical insights into SVM behavior for large datasets, which is incremental for researchers in machine learning theory.
The authors analyzed the performance of kernel least squares support vector machines (LS-SVMs) using random matrix theory in high-dimensional settings where data dimension and sample size grow proportionally, proving that the decision function becomes asymptotically normal under a Gaussian mixture model, with explicit dependencies on kernel derivatives.
This article proposes a performance analysis of kernel least squares support vector machines (LS-SVMs) based on a random matrix approach, in the regime where both the dimension of data $p$ and their number $n$ grow large at the same rate. Under a two-class Gaussian mixture model for the input data, we prove that the LS-SVM decision function is asymptotically normal with means and covariances shown to depend explicitly on the derivatives of the kernel function. This provides improved understanding along with new insights into the internal workings of SVM-type methods for large datasets.