State-space systems as dynamic generative models
This work provides a stochastic generalization of the echo state property, which is foundational for understanding dynamic generative models in machine learning and systems theory.
The paper introduces a probabilistic framework to analyze the dependence structure between input and output processes in deterministic discrete-time state-space systems, showing that under weaker contractivity conditions than in deterministic cases, these systems can act as generative models for probabilistic dependencies, even without a functional relation.
A probabilistic framework to study the dependence structure induced by deterministic discrete-time state-space systems between input and output processes is introduced. General sufficient conditions are formulated under which output processes exist and are unique once an input process has been fixed, a property that in the deterministic state-space literature is known as the echo state property. When those conditions are satisfied, the given state-space system becomes a generative model for probabilistic dependences between two sequence spaces. Moreover, those conditions guarantee that the output depends continuously on the input when using the Wasserstein metric. The output processes whose existence is proved are shown to be causal in a specific sense and to generalize those studied in purely deterministic situations. The results in this paper constitute a significant stochastic generalization of sufficient conditions for the deterministic echo state property to hold, in the sense that the stochastic echo state property can be satisfied under contractivity conditions that are strictly weaker than those in deterministic situations. This means that state-space systems can induce a purely probabilistic dependence structure between input and output sequence spaces even when there is no functional relation between those two spaces.