Lepskii Principle for Distributed Kernel Ridge Regression
This addresses a practical inconsistency in distributed learning for data analysts, offering an incremental improvement by adapting existing principles to a specific domain.
The paper tackles the challenge of parameter selection in distributed kernel ridge regression without sharing local data, proposing an adaptive method (Lep-AdaDKRR) that achieves optimal learning rates and adapts to regression function regularity, kernel decay rates, and generalization metrics.
Parameter selection without communicating local data is quite challenging in distributed learning, exhibing an inconsistency between theoretical analysis and practical application of it in tackling distributively stored data. Motivated by the recently developed Lepskii principle and non-privacy communication protocol for kernel learning, we propose a Lepskii principle to equip distributed kernel ridge regression (DKRR) and consequently develop an adaptive DKRR with Lepskii principle (Lep-AdaDKRR for short) by using a double weighted averaging synthesization scheme. We deduce optimal learning rates for Lep-AdaDKRR and theoretically show that Lep-AdaDKRR succeeds in adapting to the regularity of regression functions, effective dimension decaying rate of kernels and different metrics of generalization, which fills the gap of the mentioned inconsistency between theory and application.