On the Saturation Effect of Kernel Ridge Regression
This resolves a fundamental theoretical problem in machine learning, with implications for understanding and improving kernel methods.
The paper tackles the saturation effect in kernel ridge regression, where performance fails to reach the information theoretical lower bound for highly smooth functions, and provides a proof of a long-standing conjecture about this lower bound.
The saturation effect refers to the phenomenon that the kernel ridge regression (KRR) fails to achieve the information theoretical lower bound when the smoothness of the underground truth function exceeds certain level. The saturation effect has been widely observed in practices and a saturation lower bound of KRR has been conjectured for decades. In this paper, we provide a proof of this long-standing conjecture.