An Information-Theoretic Regularizer for Lossy Neural Image Compression
This work addresses a bottleneck in neural image compression for applications requiring efficient data storage and transmission, though it is incremental as it builds on existing compression frameworks.
The paper tackles the challenge of optimizing lossy neural image compression networks by proposing an information-theoretic regularizer based on maximizing conditional source entropy, which improves optimization and generalization, leading to reduced bits in latent representations across various structures and domains.
Lossy image compression networks aim to minimize the latent entropy of images while adhering to specific distortion constraints. However, optimizing the neural network can be challenging due to its nature of learning quantized latent representations. In this paper, our key finding is that minimizing the latent entropy is, to some extent, equivalent to maximizing the conditional source entropy, an insight that is deeply rooted in information-theoretic equalities. Building on this insight, we propose a novel structural regularization method for the neural image compression task by incorporating the negative conditional source entropy into the training objective, such that both the optimization efficacy and the model's generalization ability can be promoted. The proposed information-theoretic regularizer is interpretable, plug-and-play, and imposes no inference overheads. Extensive experiments demonstrate its superiority in regularizing the models and further squeezing bits from the latent representation across various compression structures and unseen domains.