Neural ODEs with stochastic vector field mixtures
This addresses a critical limitation in neural ODEs for modeling complex, volatile dynamics like human behavior, though it appears incremental by extending to new tasks within an existing framework.
The paper tackles the problem that neural ODE models fail at fundamental tasks, introducing two new unsolvable tasks and proposing mixtures of stochastic vector fields to solve them, with experimental demonstration on modeling volatile human behavior dynamics where baselines fail.
It was recently shown that neural ordinary differential equation models cannot solve fundamental and seemingly straightforward tasks even with high-capacity vector field representations. This paper introduces two other fundamental tasks to the set that baseline methods cannot solve, and proposes mixtures of stochastic vector fields as a model class that is capable of solving these essential problems. Dynamic vector field selection is of critical importance for our model, and our approach is to propagate component uncertainty over the integration interval with a technique based on forward filtering. We also formalise several loss functions that encourage desirable properties on the trajectory paths, and of particular interest are those that directly encourage fewer expected function evaluations. Experimentally, we demonstrate that our model class is capable of capturing the natural dynamics of human behaviour; a notoriously volatile application area. Baseline approaches cannot adequately model this problem.