Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEs
This addresses robustness issues in continuous-depth neural networks for applications requiring reliable predictions, though it appears incremental as it builds on existing neural ODE frameworks.
The paper tackles the problem of adversarial vulnerability in neural ODEs by connecting it to finite-time Lyapunov exponents (FTLEs), and proposes a training algorithm that regularizes FTLEs to improve robustness while reducing computational cost compared to full-interval regularization.
We investigate finite-time Lyapunov exponents (FTLEs), a measure for exponential separation of input perturbations, of deep neural networks within the framework of continuous-depth neural ODEs. We demonstrate that FTLEs are powerful organizers for input-output dynamics, allowing for better interpretability and the comparison of distinct model architectures. We establish a direct connection between Lyapunov exponents and adversarial vulnerability, and propose a novel training algorithm that improves robustness by FTLE regularization. The key idea is to suppress exponents far from zero in the early stage of the input dynamics. This approach enhances robustness and reduces computational cost compared to full-interval regularization, as it avoids a full ``double'' backpropagation.