Feedback Favors the Generalization of Neural ODEs
This work addresses the generalization issue for neural ODEs in applications like trajectory prediction and model predictive control, offering a novel approach inspired by biological feedback mechanisms.
The paper tackles the generalization problem of neural ODEs in continuous-time prediction tasks by introducing feedback neural networks, which use feedback loops to correct latent dynamics, resulting in significant improvements in unseen scenarios without losing accuracy on previous tasks.
The well-known generalization problem hinders the application of artificial neural networks in continuous-time prediction tasks with varying latent dynamics. In sharp contrast, biological systems can neatly adapt to evolving environments benefiting from real-time feedback mechanisms. Inspired by the feedback philosophy, we present feedback neural networks, showing that a feedback loop can flexibly correct the learned latent dynamics of neural ordinary differential equations (neural ODEs), leading to a prominent generalization improvement. The feedback neural network is a novel two-DOF neural network, which possesses robust performance in unseen scenarios with no loss of accuracy performance on previous tasks.} A linear feedback form is presented to correct the learned latent dynamics firstly, with a convergence guarantee. Then, domain randomization is utilized to learn a nonlinear neural feedback form. Finally, extensive tests including trajectory prediction of a real irregular object and model predictive control of a quadrotor with various uncertainties, are implemented, indicating significant improvements over state-of-the-art model-based and learning-based methods.