Off-the-grid: Fast and Effective Hyperparameter Search for Kernel Clustering
This work addresses a domain-specific challenge in unsupervised learning by improving hyperparameter search for kernel clustering, offering an incremental advance over existing heuristics.
The paper tackles the problem of hyperparameter tuning for kernel k-means clustering, where kernel parameters significantly affect results but lack straightforward tuning methods like cross-validation. It proposes an alternative algorithm with efficient implementation, demonstrating its ability to efficiently reveal useful hyperparameter values.
Kernel functions are a powerful tool to enhance the $k$-means clustering algorithm via the kernel trick. It is known that the parameters of the chosen kernel function can have a dramatic impact on the result. In supervised settings, these can be tuned via cross-validation, but for clustering this is not straightforward and heuristics are usually employed. In this paper we study the impact of kernel parameters on kernel $k$-means. In particular, we derive a lower bound, tight up to constant factors, below which the parameter of the RBF kernel will render kernel $k$-means meaningless. We argue that grid search can be ineffective for hyperparameter search in this context and propose an alternative algorithm for this purpose. In addition, we offer an efficient implementation based on fast approximate exponentiation with provable quality guarantees. Our experimental results demonstrate the ability of our method to efficiently reveal a rich and useful set of hyperparameter values.