Adaptive Stopping Rule for Kernel-based Gradient Descent Algorithms
This work addresses computational efficiency in machine learning optimization, offering an incremental improvement for kernel methods.
The authors tackled the problem of determining when to stop kernel-based gradient descent algorithms by proposing an adaptive stopping rule based on empirical effective dimension, which achieves optimal learning rates and provides a sharp bound on iteration count to reduce computational cost.
In this paper, we propose an adaptive stopping rule for kernel-based gradient descent (KGD) algorithms. We introduce the empirical effective dimension to quantify the increments of iterations in KGD and derive an implementable early stopping strategy. We analyze the performance of the adaptive stopping rule in the framework of learning theory. Using the recently developed integral operator approach, we rigorously prove the optimality of the adaptive stopping rule in terms of showing the optimal learning rates for KGD equipped with this rule. Furthermore, a sharp bound on the number of iterations in KGD equipped with the proposed early stopping rule is also given to demonstrate its computational advantage.