Generative Latent Video Compression
This addresses perceptual quality issues in video compression for applications like streaming, though it is incremental as it builds on prior neural codecs with architectural innovations.
The paper tackled the challenge of balancing rate-distortion-perception tradeoffs in neural video compression, which often causes flickering artifacts, by proposing Generative Latent Video Compression (GLVC) that achieves state-of-the-art performance in DISTS and LPIPS metrics and rivals latest neural codecs at nearly half their rate in user studies.
Perceptual optimization is widely recognized as essential for neural compression, yet balancing the rate-distortion-perception tradeoff remains challenging. This difficulty is especially pronounced in video compression, where frame-wise quality fluctuations often cause perceptually optimized neural video codecs to suffer from flickering artifacts. In this paper, inspired by the success of latent generative models, we present Generative Latent Video Compression (GLVC), an effective framework for perceptual video compression. GLVC employs a pretrained continuous tokenizer to project video frames into a perceptually aligned latent space, thereby offloading perceptual constraints from the rate-distortion optimization. We redesign the codec architecture explicitly for the latent domain, drawing on extensive insights from prior neural video codecs, and further equip it with innovations such as unified intra/inter coding and a recurrent memory mechanism. Experimental results across multiple benchmarks show that GLVC achieves state-of-the-art performance in terms of DISTS and LPIPS metrics. Notably, our user study confirms GLVC rivals the latest neural video codecs at nearly half their rate while maintaining stable temporal coherence, marking a step toward practical perceptual video compression.