Towards Recurrent Autoregressive Flow Models
This work addresses a challenge in machine learning for representing complex stochastic processes, though it appears incremental as it builds on existing normalizing flow and recurrent network techniques.
The authors tackled the problem of modeling stochastic processes with non-stationary distributions by proposing Recurrent Autoregressive Flows, which combine normalizing flows with recurrent neural networks to define conditional distributions in sequences, and demonstrated its effectiveness on three complex stochastic processes.
Stochastic processes generated by non-stationary distributions are difficult to represent with conventional models such as Gaussian processes. This work presents Recurrent Autoregressive Flows as a method toward general stochastic process modeling with normalizing flows. The proposed method defines a conditional distribution for each variable in a sequential process by conditioning the parameters of a normalizing flow with recurrent neural connections. Complex conditional relationships are learned through the recurrent network parameters. In this work, we present an initial design for a recurrent flow cell and a method to train the model to match observed empirical distributions. We demonstrate the effectiveness of this class of models through a series of experiments in which models are trained on three complex stochastic processes. We highlight the shortcomings of our current formulation and suggest some potential solutions.