Parameter-Efficient Generative Modeling with Controlled Vector Fields
This work proposes a novel geometric approach to generative modeling that reduces parameter count, offering a potential alternative to standard high-dimensional vector field parameterizations, though it is currently limited to synthetic data.
The paper introduces a parameter-efficient generative modeling framework that constructs expressive flows using a small set of fixed vector fields and learned scalar controls, reducing the number of learned parameters. Proof-of-concept experiments on synthetic distributions demonstrate the feasibility of the approach.
We introduce a continuous-time generative modeling framework, motivated by the Chow-Rashevskii theorem, that builds expressive flows from a small set of fixed vector fields and learned scalar controls. Instead of learning an unconstrained high-dimensional vector field, our framework constructs the velocity by modulating fixed vector fields with learned scalar control functions. When the fixed fields are bracket-generating, their Lie algebra spans the ambient space, providing a mechanism for expressive transport with only a small number of learned control channels and offering a parameter-efficient geometric alternative to standard vector-field parameterizations. This decoupled formulation yields a structured and interpretable generative model in which the number of learned scalar output channels can be chosen independently of the ambient dimension. We formulate an expressivity principle showing that, under suitable controllability and well-posedness assumptions, such controlled flows can transport a source distribution to a target distribution. We train the resulting model using a continuous-normalizing-flow likelihood objective and present proof-of-concept experiments on synthetic distributions.