A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
This work addresses optimization challenges in machine learning and engineering for scenarios with stochastic objectives and constraints, though it appears incremental as it combines existing trust-region, interior-point, and stochastic methods.
The authors tackled the problem of solving optimization problems with stochastic objectives and deterministic nonlinear constraints by proposing a trust-region interior-point stochastic sequential quadratic programming method, establishing global almost-sure convergence to first-order stationary points under standard assumptions and demonstrating practical performance on CUTEst test set and logistic regression problems.
In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their stochastic estimates can be constructed. In particular, at each iteration our method builds stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.