An Outer-approximation Guided Optimization Approach for Constrained Neural Network Inverse Problems
This work addresses constrained inverse problems for neural networks, which is incremental as it builds on existing optimization techniques for specific activation functions.
The paper tackles the problem of finding optimal inputs for trained neural networks with ReLU activations to achieve desired outputs under constraints, proposing an outer-approximation guided optimization method that outperforms a projected gradient method in computational experiments.
This paper discusses an outer-approximation guided optimization method for constrained neural network inverse problems with rectified linear units. The constrained neural network inverse problems refer to an optimization problem to find the best set of input values of a given trained neural network in order to produce a predefined desired output in presence of constraints on input values. This paper analyzes the characteristics of optimal solutions of neural network inverse problems with rectified activation units and proposes an outer-approximation algorithm by exploiting their characteristics. The proposed outer-approximation guided optimization comprises primal and dual phases. The primal phase incorporates neighbor curvatures with neighbor outer-approximations to expedite the process. The dual phase identifies and utilizes the structure of local convex regions to improve the convergence to a local optimal solution. At last, computation experiments demonstrate the superiority of the proposed algorithm compared to a projected gradient method.