Gradient Estimation with Simultaneous Perturbation and Compressive Sensing
This work addresses gradient estimation for high-dimensional optimization, but it appears incremental as it combines existing techniques without claiming major breakthroughs.
The paper tackles the problem of gradient estimation in high-dimensional spaces where gradients are nearly sparse, proposing a method that combines Simultaneous Perturbation Stochastic Approximation with compressive sensing to reduce function evaluations. Simulations demonstrate applications in estimating gradient outer product matrices and optimization problems.
This paper aims at achieving a "good" estimator for the gradient of a function on a high-dimensional space. Often such functions are not sensitive in all coordinates and the gradient of the function is almost sparse. We propose a method for gradient estimation that combines ideas from Spall's Simultaneous Perturbation Stochastic Approximation with compressive sensing. The aim is to obtain "good" estimator without too many function evaluations. Application to estimating gradient outer product matrix as well as standard optimization problems are illustrated via simulations.