On the advantages of stochastic encoders
This addresses a theoretical gap in neural compression and rate-distortion theory, though it is incremental as it provides only an illustrative example.
The paper tackles the problem of whether stochastic encoders can outperform deterministic encoders, showing through a toy example that they significantly outperform the best deterministic encoders, particularly in the regime of perfect perceptual quality.
Stochastic encoders have been used in rate-distortion theory and neural compression because they can be easier to handle. However, in performance comparisons with deterministic encoders they often do worse, suggesting that noise in the encoding process may generally be a bad idea. It is poorly understood if and when stochastic encoders do better than deterministic encoders. In this paper we provide one illustrative example which shows that stochastic encoders can significantly outperform the best deterministic encoders. Our toy example suggests that stochastic encoders may be particularly useful in the regime of "perfect perceptual quality".