DFORM: Diffeomorphic vector field alignment for assessing dynamics across learned models
This provides a method for researchers in computational neuroscience and machine learning to assess and compare the dynamics of learned models, though it is incremental as it builds on existing concepts of orbital equivalence.
The authors tackled the challenge of comparing learned dynamics across nonlinear models like RNNs, which lack coordinate equivalence, by proposing DFORM, a framework that learns diffeomorphic transformations to align vector fields and measure orbital similarity. They demonstrated its application on neuroscience models, revealing functional similarities despite differences in attractor landscapes.
Dynamical system models such as Recurrent Neural Networks (RNNs) have become increasingly popular as hypothesis-generating tools in scientific research. Evaluating the dynamics in such networks is key to understanding their learned generative mechanisms. However, comparison of learned dynamics across models is challenging due to their inherent nonlinearity and because a priori there is no enforced equivalence of their coordinate systems. Here, we propose the DFORM (Diffeomorphic vector field alignment for comparing dynamics across learned models) framework. DFORM learns a nonlinear coordinate transformation which provides a continuous, maximally one-to-one mapping between the trajectories of learned models, thus approximating a diffeomorphism between them. The mismatch between DFORM-transformed vector fields defines the orbital similarity between two models, thus providing a generalization of the concepts of smooth orbital and topological equivalence. As an example, we apply DFORM to models trained on a canonical neuroscience task, showing that learned dynamics may be functionally similar, despite overt differences in attractor landscapes.