Latent Regression Bayesian Network for Data Representation
This work addresses a bottleneck in learning deep generative models for researchers in machine learning, but it is incremental as it builds on existing methods with specific improvements.
The paper tackled the problem of intractable inference in deep directed generative models by proposing a method that preserves correlations among latent variables, using conditional pseudo-likelihood and hard EM, and demonstrated effectiveness in data representation and reconstruction on benchmark datasets.
Deep directed generative models have attracted much attention recently due to their expressive representation power and the ability of ancestral sampling. One major difficulty of learning directed models with many latent variables is the intractable inference. To address this problem, most existing algorithms make assumptions to render the latent variables independent of each other, either by designing specific priors, or by approximating the true posterior using a factorized distribution. We believe the correlations among latent variables are crucial for faithful data representation. Driven by this idea, we propose an inference method based on the conditional pseudo-likelihood that preserves the dependencies among the latent variables. For learning, we propose to employ the hard Expectation Maximization (EM) algorithm, which avoids the intractability of the traditional EM by max-out instead of sum-out to compute the data likelihood. Qualitative and quantitative evaluations of our model against state of the art deep models on benchmark datasets demonstrate the effectiveness of the proposed algorithm in data representation and reconstruction.