The Error Probability of Random Fourier Features is Dimensionality Independent
This work addresses the scalability of kernel methods in machine learning by providing dimension-independent error bounds, which is significant for practitioners dealing with high-dimensional data, though it builds incrementally on prior theoretical analysis.
The paper tackles the problem of reconstructing kernel matrices using Random Fourier Features for the Gaussian kernel, showing that the error probability is at most O(R^{2/3} exp(-D)) and providing a lower bound of Ω((1-exp(-R^2)) exp(-D)), with the key result being that this error is independent of data dimensionality.
We show that the error probability of reconstructing kernel matrices from Random Fourier Features for the Gaussian kernel function is at most $\mathcal{O}(R^{2/3} \exp(-D))$, where $D$ is the number of random features and $R$ is the diameter of the data domain. We also provide an information-theoretic method-independent lower bound of $Ω((1-\exp(-R^2)) \exp(-D))$. Compared to prior work, we are the first to show that the error probability for random Fourier features is independent of the dimensionality of data points. As applications of our theory, we obtain dimension-independent bounds for kernel ridge regression and support vector machines.