Parameter-Efficient Neural CDEs via Implicit Function Jacobians
This addresses a scalability issue for researchers and practitioners using Neural CDEs in sequence modeling, though it appears incremental as it builds directly on existing methods.
The paper tackles the high parameter count in Neural Controlled Differential Equations (Neural CDEs) for temporal sequence analysis by proposing a parameter-efficient alternative, achieving a significant reduction in parameters while maintaining a logical analogy to continuous RNNs.
Neural Controlled Differential Equations (Neural CDEs, NCDEs) are a unique branch of methods, specifically tailored for analysing temporal sequences. However, they come with drawbacks, the main one being the number of parameters, required for the method's operation. In this paper, we propose an alternative, parameter-efficient look at Neural CDEs. It requires much fewer parameters, while also presenting a very logical analogy as the "Continuous RNN", which the Neural CDEs aspire to.