A Multi-parameter Updating Fourier Online Gradient Descent Algorithm for Large-scale Nonlinear Classification
This work addresses challenges in support vector machines for large-scale nonlinear classification, offering an incremental improvement over existing online methods.
The paper tackled the problem of large-scale nonlinear classification by addressing shortcomings in online random Fourier feature map algorithms, such as fixed random directions and high dimensionality, and proposed a multi-parameter updating Fourier online gradient descent algorithm (MPU-FOGD) that achieved better test accuracy on benchmark datasets compared to state-of-the-art methods.
Large scale nonlinear classification is a challenging task in the field of support vector machine. Online random Fourier feature map algorithms are very important methods for dealing with large scale nonlinear classification problems. The main shortcomings of these methods are as follows: (1) Since only the hyperplane vector is updated during learning while the random directions are fixed, there is no guarantee that these online methods can adapt to the change of data distribution when the data is coming one by one. (2) The dimension of the random direction is often higher for obtaining better classification accuracy, which results in longer test time. In order to overcome these shortcomings, a multi-parameter updating Fourier online gradient descent algorithm (MPU-FOGD) is proposed for large-scale nonlinear classification problems based on a novel random feature map. In the proposed method, the suggested random feature map has lower dimension while the multi-parameter updating strategy can guarantee the learning model can better adapt to the change of data distribution when the data is coming one by one. Theoretically, it is proved that compared with the existing random Fourier feature maps, the proposed random feature map can give a tighter error bound. Empirical studies on several benchmark data sets demonstrate that compared with the state-of-the-art online random Fourier feature map methods, the proposed MPU-FOGD can obtain better test accuracy.