Joint-stochastic-approximation Random Fields with Application to Semi-supervised Learning
This addresses problems in semi-supervised learning for researchers using generative models, but it is incremental as it builds on existing undirected models.
The paper tackled mode missing and mode covering issues in deep generative models for semi-supervised learning, and conflicts between classification and generation, by proposing joint-stochastic-approximation random fields (JRFs), which achieved classification results comparable to state-of-the-art methods on MNIST, SVHN, and CIFAR-10 while performing good generation.
Our examination of deep generative models (DGMs) developed for semi-supervised learning (SSL), mainly GANs and VAEs, reveals two problems. First, mode missing and mode covering phenomenons are observed in genertion with GANs and VAEs. Second, there exists an awkward conflict between good classification and good generation in SSL by employing directed generative models. To address these problems, we formally present joint-stochastic-approximation random fields (JRFs) -- a new family of algorithms for building deep undirected generative models, with application to SSL. It is found through synthetic experiments that JRFs work well in balancing mode covering and mode missing, and match the empirical data distribution well. Empirically, JRFs achieve good classification results comparable to the state-of-art methods on widely adopted datasets -- MNIST, SVHN, and CIFAR-10 in SSL, and simultaneously perform good generation.