Large Scale Kernel Learning using Block Coordinate Descent
This work addresses scalability issues in kernel methods for machine learning practitioners, though it is incremental as it applies an existing optimization technique to kernel problems.
The paper tackled large-scale kernel learning by using distributed block coordinate descent to efficiently solve kernel regression and classification with millions of data points, finding that the Nyström method generally achieves better statistical accuracy than random features but requires more optimization iterations.
We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points. Armed with this capability, we conduct a thorough comparison between the full kernel, the Nyström method, and random features on three large classification tasks from various domains. Our results suggest that the Nyström method generally achieves better statistical accuracy than random features, but can require significantly more iterations of optimization. Lastly, we derive new rates for block coordinate descent which support our experimental findings when specialized to kernel methods.