Convergence for adaptive resampling of random Fourier features
Provides theoretical convergence guarantees for adaptive frequency sampling in random Fourier features, addressing a key challenge for high-dimensional machine learning.
This work proves convergence of a data-adaptive resampling method for random Fourier features, showing asymptotic optimality as data and nodes grow. Numerical results confirm the analysis for regression and classification.
The machine learning random Fourier feature method for data in high dimension is computationally and theoretically attractive since the optimization is based on a convex standard least squares problem and independent sampling of Fourier frequencies. The challenge is to sample the Fourier frequencies well. This work proves convergence of a data adaptive method based on resampling the frequencies asymptotically optimally, as the number of nodes and amount of data tend to infinity. Numerical results based on resampling and adaptive random walk steps together with approximations of the least squares problem by conjugate gradient iterations confirm the analysis for regression and classification problems.