Semi-Implicit Neural Ordinary Differential Equations
This addresses efficiency and robustness issues for researchers and practitioners in graph learning and scientific machine learning, offering a novel method for challenging neural ODE training.
The paper tackles the stability limitations of classical neural ODEs in stiff learning problems like graph learning and scientific machine learning, resulting in a semi-implicit neural ODE approach that outperforms existing methods on applications such as graph classification and learning complex dynamical systems.
Classical neural ODEs trained with explicit methods are intrinsically limited by stability, crippling their efficiency and robustness for stiff learning problems that are common in graph learning and scientific machine learning. We present a semi-implicit neural ODE approach that exploits the partitionable structure of the underlying dynamics. Our technique leads to an implicit neural network with significant computational advantages over existing approaches because of enhanced stability and efficient linear solves during time integration. We show that our approach outperforms existing approaches on a variety of applications including graph classification and learning complex dynamical systems. We also demonstrate that our approach can train challenging neural ODEs where both explicit methods and fully implicit methods are intractable.