Data Generation as Sequential Decision Making
This work addresses data imputation, a common issue in data analysis, by applying generative modeling techniques, but it is incremental as it builds on existing sequential decision-making frameworks.
The paper tackles the problem of data imputation by framing it as a sequential decision-making process, formulating it as a Markov Decision Process (MDP) and training neural network models using guided policy search, resulting in models that learn effective policies across varying difficulty levels and multiple datasets.
We connect a broad class of generative models through their shared reliance on sequential decision making. Motivated by this view, we develop extensions to an existing model, and then explore the idea further in the context of data imputation -- perhaps the simplest setting in which to investigate the relation between unconditional and conditional generative modelling. We formulate data imputation as an MDP and develop models capable of representing effective policies for it. We construct the models using neural networks and train them using a form of guided policy search. Our models generate predictions through an iterative process of feedback and refinement. We show that this approach can learn effective policies for imputation problems of varying difficulty and across multiple datasets.