Normalized Gradients for All
arXiv:2308.05621v123 citationsh-index: 42
Originality Synthesis-oriented
AI Analysis
This is an incremental improvement for optimization researchers, building directly on prior work.
The paper tackles the problem of adapting to Hölder smoothness in optimization by using normalized gradients in a black-box manner, achieving a bound that depends on a novel notion of local Hölder smoothness.
In this short note, I show how to adapt to Hölder smoothness using normalized gradients in a black-box way. Moreover, the bound will depend on a novel notion of local Hölder smoothness. The main idea directly comes from Levy [2017].