Reducing The Mismatch Between Marginal and Learned Distributions in Neural Video Compression
This addresses a specific bottleneck in neural video compression for improving efficiency, but it is incremental as it builds on existing methods.
The paper tackled the amortization gap problem in neural video compression, where a mismatch between latent's marginal and learned distributions increases file size, and proposed a generic method that improved compression by 2% to 5% without affecting reconstruction quality.
During the last four years, we have witnessed the success of end-to-end trainable models for image compression. Compared to decades of incremental work, these machine learning (ML) techniques learn all the components of the compression technique, which explains their actual superiority. However, end-to-end ML models have not yet reached the performance of traditional video codecs such as VVC. Possible explanations can be put forward: lack of data to account for the temporal redundancy, or inefficiency of latent's density estimation in the neural model. The latter problem can be defined by the discrepancy between the latent's marginal distribution and the learned prior distribution. This mismatch, known as amortization gap of entropy model, enlarges the file size of compressed data. In this paper, we propose to evaluate the amortization gap for three state-of-the-art ML video compression methods. Second, we propose an efficient and generic method to solve the amortization gap and show that it leads to an improvement between $2\%$ to $5\%$ without impacting reconstruction quality.