Signature Reconstruction from Randomized Signatures
This work addresses a theoretical problem in machine learning for understanding feature extraction from curves, but it appears incremental as it builds on existing results from Lie algebra theory.
The paper investigates the extent to which signature features of curves can be reconstructed from controlled ordinary differential equations with random vector fields, showing that the number of reconstructable features is exponential in the hidden dimension for specific neural vector fields and characterizing a general condition for reconstruction up to a fixed order.
Controlled ordinary differential equations driven by continuous bounded variation curves can be considered a continuous time analogue of recurrent neural networks for the construction of expressive features of the input curves. We ask up to which extent well known signature features of such curves can be reconstructed from controlled ordinary differential equations with (untrained) random vector fields. The answer turns out to be algebraically involved, but essentially the number of signature features, which can be reconstructed from the non-linear flow of the controlled ordinary differential equation, is exponential in its hidden dimension, when the vector fields are chosen to be neural with depth two. Moreover, we characterize a general linear independence condition on arbitrary vector fields, under which the signature features up to some fixed order can always be reconstructed. Algebraically speaking this complements in a quantitative manner several well known results from the theory of Lie algebras of vector fields and puts them in a context of machine learning.