Improving Inference for Neural Image Compression
This work addresses efficiency and quality issues in image compression for applications like storage and transmission, representing an incremental improvement over existing methods.
The paper tackled performance limitations in neural image compression by identifying three approximation gaps in hierarchical variational autoencoders and proposed remedies including iterative inference and bits-back coding, achieving new state-of-the-art results on lossy image compression benchmarks.
We consider the problem of lossy image compression with deep latent variable models. State-of-the-art methods build on hierarchical variational autoencoders (VAEs) and learn inference networks to predict a compressible latent representation of each data point. Drawing on the variational inference perspective on compression, we identify three approximation gaps which limit performance in the conventional approach: an amortization gap, a discretization gap, and a marginalization gap. We propose remedies for each of these three limitations based on ideas related to iterative inference, stochastic annealing for discrete optimization, and bits-back coding, resulting in the first application of bits-back coding to lossy compression. In our experiments, which include extensive baseline comparisons and ablation studies, we achieve new state-of-the-art performance on lossy image compression using an established VAE architecture, by changing only the inference method.