Combining Generative and Discriminative Models for Hybrid Inference
This work addresses the issue of poor approximation in data generating processes for researchers in machine learning and statistics, though it appears incremental as it builds on existing graphical and learned inference methods.
The authors tackled the problem of suboptimal estimation in graphical models by proposing a hybrid model that combines graphical inference with a learned inverse model, structured as a graph neural network and formulated as a recurrent neural network, and showed that it estimates the trajectory of a noisy chaotic Lorenz Attractor more accurately than isolated methods.
A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimation. The subtleties of the generative process are however captured in the data itself and we can `learn to infer', that is, learn a direct mapping from observations to explanatory latent variables. In this work we propose a hybrid model that combines graphical inference with a learned inverse model, which we structure as in a graph neural network, while the iterative algorithm as a whole is formulated as a recurrent neural network. By using cross-validation we can automatically balance the amount of work performed by graphical inference versus learned inference. We apply our ideas to the Kalman filter, a Gaussian hidden Markov model for time sequences, and show, among other things, that our model can estimate the trajectory of a noisy chaotic Lorenz Attractor much more accurately than either the learned or graphical inference run in isolation.