Conditional Image Generation with PixelCNN Decoders
This work addresses the problem of generating high-quality, varied images under specific conditions for applications in computer vision and graphics, representing an incremental improvement over prior methods.
The authors tackled conditional image generation by introducing a new density model based on PixelCNN, which can be conditioned on vectors like labels or embeddings to produce diverse, realistic images, such as distinct scenes from ImageNet or varied portraits from a single face image, while matching PixelRNN's log-likelihood on ImageNet with reduced computational cost.
This work explores conditional image generation with a new image density model based on the PixelCNN architecture. The model can be conditioned on any vector, including descriptive labels or tags, or latent embeddings created by other networks. When conditioned on class labels from the ImageNet database, the model is able to generate diverse, realistic scenes representing distinct animals, objects, landscapes and structures. When conditioned on an embedding produced by a convolutional network given a single image of an unseen face, it generates a variety of new portraits of the same person with different facial expressions, poses and lighting conditions. We also show that conditional PixelCNN can serve as a powerful decoder in an image autoencoder. Additionally, the gated convolutional layers in the proposed model improve the log-likelihood of PixelCNN to match the state-of-the-art performance of PixelRNN on ImageNet, with greatly reduced computational cost.