Robust Optimization Framework for Training Shallow Neural Networks Using Reachability Method
This work addresses robustness in neural network training for applications like robotics, but it is incremental as it builds on existing reachability and robust optimization methods.
The paper tackles the problem of training shallow neural networks to be robust against input perturbations by developing a robust optimization framework using reachability analysis, resulting in improved robustness at the cost of some training accuracy loss, as demonstrated on a robot arm model learning example.
In this paper, a robust optimization framework is developed to train shallow neural networks based on reachability analysis of neural networks. To characterize noises of input data, the input training data is disturbed in the description of interval sets. Interval-based reachability analysis is then performed for the hidden layer. With the reachability analysis results, a robust optimization training method is developed in the framework of robust least-square problems. Then, the developed robust least-square problem is relaxed to a semidefinite programming problem. It has been shown that the developed robust learning method can provide better robustness against perturbations at the price of loss of training accuracy to some extent. At last, the proposed method is evaluated on a robot arm model learning example.