Stochastic Optimization with Random Search
This work addresses stochastic optimization problems where only noisy evaluations are available, offering incremental improvements to random search methods.
The paper revisits random search for stochastic optimization under noisy function evaluations, showing it works under weaker smoothness assumptions and that stronger assumptions enable improved guarantees, with a variance-reduced variant in the finite-sum setting accelerating convergence.
We revisit random search for stochastic optimization, where only noisy function evaluations are available. We show that the method works under weaker smoothness assumptions than previously considered, and that stronger assumptions enable improved guarantees. In the finite-sum setting, we design a variance-reduced variant that leverages multiple samples to accelerate convergence. Our analysis relies on a simple translation invariance property, which provides a principled way to balance noise and reduce variance.