A New Algorithm for Tessellated Kernel Learning
This work addresses the problem of scaling kernel learning algorithms for researchers and practitioners dealing with larger datasets, though it is incremental as it builds on the existing TK framework.
The paper tackles the scalability and applicability limitations of previous Tessellated Kernel (TK) optimization methods by introducing a new two-step algorithm that scales to 10,000 data points and extends to regression, demonstrating significant performance improvements over Neural Nets and SimpleMKL on benchmark data with similar computation time.
The accuracy and complexity of machine learning algorithms based on kernel optimization are limited by the set of kernels over which they are able to optimize. An ideal set of kernels should: admit a linear parameterization (for tractability); be dense in the set of all kernels (for robustness); be universal (for accuracy). The recently proposed Tesselated Kernels (TKs) is currently the only known class which meets all three criteria. However, previous algorithms for optimizing TKs were limited to classification and relied on Semidefinite Programming (SDP) - limiting them to relatively small datasets. By contrast, the 2-step algorithm proposed here scales to 10,000 data points and extends to the regression problem. Furthermore, when applied to benchmark data, the algorithm demonstrates significant improvement in performance over Neural Nets and SimpleMKL with similar computation time.