Forward sensitivity analysis for contracting stochastic systems
Provides theoretical foundations and a practical estimator for gradient computation in a class of stochastic systems, which is incremental for the control and optimization community.
This work establishes conditions for differentiability of stationary costs in contracting stochastic systems and derives a gradient estimator generalizing forward sensitivity analysis from deterministic systems. The method is applied to examples including neural network models.
In this work we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance, that guarantee differentiability of stationary costs. Then we show how to estimate the derivatives, deriving an estimator that can be seen as a generalization of the forward sensitivity analysis method used in deterministic systems. We apply the results to examples, including a neural network model.