Constrained Stochastic Nonconvex Optimization with State-dependent Markov Data
This addresses optimization challenges in machine learning applications like strategic classification and reinforcement learning, but it is incremental as it builds on existing methods with specific theoretical improvements.
The paper tackles constrained nonconvex stochastic optimization with state-dependent Markov data, establishing that projection-based and projection-free algorithms require O(1/ε^2.5) stochastic first-order oracle calls and O(1/ε^5.5) linear minimization oracle calls, respectively, to find an ε-stationary point.
We study stochastic optimization algorithms for constrained nonconvex stochastic optimization problems with Markovian data. In particular, we focus on the case when the transition kernel of the Markov chain is state-dependent. Such stochastic optimization problems arise in various machine learning problems including strategic classification and reinforcement learning. For this problem, we study both projection-based and projection-free algorithms. In both cases, we establish that the number of calls to the stochastic first-order oracle to obtain an appropriately defined $ε$-stationary point is of the order $\mathcal{O}(1/ε^{2.5})$. In the projection-free setting we additionally establish that the number of calls to the linear minimization oracle is of order $\mathcal{O}(1/ε^{5.5})$. We also empirically demonstrate the performance of our algorithm on the problem of strategic classification with neural networks.