Augmented Neural ODEs
This addresses a fundamental problem in machine learning for researchers and practitioners using neural ODEs, offering a novel solution with broad implications.
The authors tackled the limitations of Neural ODEs in representing certain functions due to topology preservation, and introduced Augmented Neural ODEs, which are more expressive, empirically more stable, generalize better, and have lower computational cost.
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive models, are empirically more stable, generalize better and have a lower computational cost than Neural ODEs.