PixelPyramids: Exact Inference Models from Lossless Image Pyramids
This addresses the problem of scaling exact inference models to high-resolution images for researchers and practitioners in computer vision, representing a strong incremental advance.
The paper tackles the computational expense and resolution limitations of autoregressive models for image density estimation by proposing PixelPyramids, a block-autoregressive method using lossless pyramid decomposition, which achieves state-of-the-art results, improving density estimates on CelebA-HQ 1024x1024 to ~44% of the baseline with faster sampling.
Autoregressive models are a class of exact inference approaches with highly flexible functional forms, yielding state-of-the-art density estimates for natural images. Yet, the sequential ordering on the dimensions makes these models computationally expensive and limits their applicability to low-resolution imagery. In this work, we propose Pixel-Pyramids, a block-autoregressive approach employing a lossless pyramid decomposition with scale-specific representations to encode the joint distribution of image pixels. Crucially, it affords a sparser dependency structure compared to fully autoregressive approaches. Our PixelPyramids yield state-of-the-art results for density estimation on various image datasets, especially for high-resolution data. For CelebA-HQ 1024 x 1024, we observe that the density estimates (in terms of bits/dim) are improved to ~44% of the baseline despite sampling speeds superior even to easily parallelizable flow-based models.