Semi-Supervised Generative Learning via Latent Space Distribution Matching
This work provides a new semi-supervised generative modeling framework that improves generation quality by effectively leveraging unpaired data, which is beneficial for researchers and practitioners in generative AI dealing with limited paired datasets.
This paper introduces Latent Space Distribution Matching (LSDM), a semi-supervised generative framework that learns a low-dimensional latent space from both paired and unpaired data, then performs joint distribution matching in this space using only paired data. This method reduces reliance on scarce paired samples and enables fast one-step generation, demonstrating enhanced geometric fidelity in generated outputs on real-world image tasks.
We introduce Latent Space Distribution Matching (LSDM), a novel framework for semi-supervised generative modeling of conditional distributions. LSDM operates in two stages: (i) learning a low-dimensional latent space from both paired and unpaired data, and (ii) performing joint distribution matching in this space via the 1-Wasserstein distance, using only paired data. This two-step approach minimizes an upper bound on the 1-Wasserstein distance between joint distributions, reducing reliance on scarce paired samples while enabling fast one-step generation. Theoretically, we establish non-asymptotic error bounds and demonstrate a key benefit of unpaired data: enhanced geometric fidelity in generated outputs. Furthermore, by extending the scope of its two core steps, LSDM provides a coherent statistical perspective that connects to a broad class of latent-space approaches. Notably, Latent Diffusion Models (LDMs) can be viewed as a variant of LSDM, in which joint distribution matching is achieved indirectly via score matching. Consequently, our results also provide theoretical insights into the consistency of LDMs. Empirical evaluations on real-world image tasks, including class-conditional generation and image super-resolution, demonstrate the effectiveness of LSDM in leveraging unpaired data to enhance generation quality.