Conditional Hallucinations for Image Compression
This addresses image compression challenges for users sensitive to semantic changes, though it is incremental as it builds on existing perceptual loss techniques.
The paper tackles the problem of balancing hallucination and out-of-distribution samples in lossy image compression by proposing a method that dynamically adjusts hallucination based on content, resulting in a model that outperforms state-of-the-art methods.
In lossy image compression, models face the challenge of either hallucinating details or generating out-of-distribution samples due to the information bottleneck. This implies that at times, introducing hallucinations is necessary to generate in-distribution samples. The optimal level of hallucination varies depending on image content, as humans are sensitive to small changes that alter the semantic meaning. We propose a novel compression method that dynamically balances the degree of hallucination based on content. We collect data and train a model to predict user preferences on hallucinations. By using this prediction to adjust the perceptual weight in the reconstruction loss, we develop a Conditionally Hallucinating compression model (ConHa) that outperforms state-of-the-art image compression methods. Code and images are available at https://polybox.ethz.ch/index.php/s/owS1k5JYs4KD4TA.