NVC-1B: A Large Neural Video Coding Model
This work addresses video compression for applications requiring efficient storage and transmission, representing an incremental advancement by scaling up existing neural coding components.
The authors tackled the problem of scaling neural video coding models by exploring how model size and architecture affect compression performance, resulting in NVC-1B, a model with over 1 billion parameters that achieves state-of-the-art compression efficiency and significant improvement over a small baseline.
The emerging large models have achieved notable progress in the fields of natural language processing and computer vision. However, large models for neural video coding are still unexplored. In this paper, we try to explore how to build a large neural video coding model. Based on a small baseline model, we gradually scale up the model sizes of its different coding parts, including the motion encoder-decoder, motion entropy model, contextual encoder-decoder, contextual entropy model, and temporal context mining module, and analyze the influence of model sizes on video compression performance. Then, we explore to use different architectures, including CNN, mixed CNN-Transformer, and Transformer architectures, to implement the neural video coding model and analyze the influence of model architectures on video compression performance. Based on our exploration results, we design the first neural video coding model with more than 1 billion parameters -- NVC-1B. Experimental results show that our proposed large model achieves a significant video compression performance improvement over the small baseline model, and represents the state-of-the-art compression efficiency. We anticipate large models may bring up the video coding technologies to the next level.