Towards Robust Neural Networks via Close-loop Control
This work addresses the problem of improving the robustness of neural networks against data perturbations for users of deep learning models, offering an incremental improvement through a novel control-theoretic approach.
This paper tackles the vulnerability of deep neural networks to perturbations by introducing a close-loop control method. This method adaptively generates control signals for perturbed data, improving robustness against various data perturbations while maintaining performance on clean data and further enhancing robustly trained networks.
Despite their success in massive engineering applications, deep neural networks are vulnerable to various perturbations due to their black-box nature. Recent study has shown that a deep neural network can misclassify the data even if the input data is perturbed by an imperceptible amount. In this paper, we address the robustness issue of neural networks by a novel close-loop control method from the perspective of dynamic systems. Instead of modifying the parameters in a fixed neural network architecture, a close-loop control process is added to generate control signals adaptively for the perturbed or corrupted data. We connect the robustness of neural networks with optimal control using the geometrical information of underlying data to design the control objective. The detailed analysis shows how the embedding manifolds of state trajectory affect error estimation of the proposed method. Our approach can simultaneously maintain the performance on clean data and improve the robustness against many types of data perturbations. It can also further improve the performance of robustly trained neural networks against different perturbations. To the best of our knowledge, this is the first work that improves the robustness of neural networks with close-loop control.