McKernel: A Library for Approximate Kernel Expansions in Log-linear Time
This provides a scalable alternative to deep learning for non-linear classification, though it is incremental as it builds on existing kernel approximation techniques.
The paper tackles the problem of scaling kernel methods for large-scale machine learning by introducing McKernel, a library that computes approximate kernel expansions in log-linear time using optimized Fast Walsh Hadamard transforms, achieving compelling speed and outperforming state-of-the-art methods on datasets like MNIST and FASHION MNIST.
McKernel introduces a framework to use kernel approximates in the mini-batch setting with Stochastic Gradient Descent (SGD) as an alternative to Deep Learning. Based on Random Kitchen Sinks [Rahimi and Recht 2007], we provide a C++ library for Large-scale Machine Learning. It contains a CPU optimized implementation of the algorithm in [Le et al. 2013], that allows the computation of approximated kernel expansions in log-linear time. The algorithm requires to compute the product of matrices Walsh Hadamard. A cache friendly Fast Walsh Hadamard that achieves compelling speed and outperforms current state-of-the-art methods has been developed. McKernel establishes the foundation of a new architecture of learning that allows to obtain large-scale non-linear classification combining lightning kernel expansions and a linear classifier. It travails in the mini-batch setting working analogously to Neural Networks. We show the validity of our method through extensive experiments on MNIST and FASHION MNIST [Xiao et al. 2017].