Curriculum Learning for Deep Generative Models with Clustering
This addresses the problem of noisy data training for generative models, offering a domain-specific improvement that is incremental in nature.
The paper tackles the challenge of training generative models like GANs on noisy data by proposing a curriculum learning algorithm based on clustering centrality, which prioritizes high-centrality data points and uses an active set for scalability. Experiments on cat and human-face data show the algorithm learns optimal generative models (e.g., ProGAN) with respect to quality metrics for noisy data, with the optimal curriculum linked to a geometric percolation critical point.
Training generative models like Generative Adversarial Network (GAN) is challenging for noisy data. A novel curriculum learning algorithm pertaining to clustering is proposed to address this issue in this paper. The curriculum construction is based on the centrality of underlying clusters in data points. The data points of high centrality takes priority of being fed into generative models during training. To make our algorithm scalable to large-scale data, the active set is devised, in the sense that every round of training proceeds only on an active subset containing a small fraction of already trained data and the incremental data of lower centrality. Moreover, the geometric analysis is presented to interpret the necessity of cluster curriculum for generative models. The experiments on cat and human-face data validate that our algorithm is able to learn the optimal generative models (e.g. ProGAN) with respect to specified quality metrics for noisy data. An interesting finding is that the optimal cluster curriculum is closely related to the critical point of the geometric percolation process formulated in the paper.