Don't Look into the Dark: Latent Codes for Pluralistic Image Inpainting
This work addresses image inpainting for computer vision applications, presenting an incremental improvement over existing methods.
The paper tackles the problem of pluralistic image inpainting with large masks by using discrete latent codes, achieving improved visual quality and diversity metrics compared to strong baselines.
We present a method for large-mask pluralistic image inpainting based on the generative framework of discrete latent codes. Our method learns latent priors, discretized as tokens, by only performing computations at the visible locations of the image. This is realized by a restrictive partial encoder that predicts the token label for each visible block, a bidirectional transformer that infers the missing labels by only looking at these tokens, and a dedicated synthesis network that couples the tokens with the partial image priors to generate coherent and pluralistic complete image even under extreme mask settings. Experiments on public benchmarks validate our design choices as the proposed method outperforms strong baselines in both visual quality and diversity metrics.