Rate-Adaptive Quantization: A Multi-Rate Codebook Adaptation for Vector Quantization-based Generative Models
This work addresses the adaptability issue in VQ-based generative models for data compression, reconstruction, and generation tasks, offering an incremental improvement by enabling multi-rate capabilities without extensive retraining.
The paper tackles the problem of fixed-rate codebooks in vector quantization-based generative models, which require retraining for new bitrates, by introducing Rate-Adaptive Quantization (RAQ) to generate variable-rate codebooks from a single baseline model, enabling flexible tradeoffs and often outperforming fixed-rate baselines across multiple rates.
Learning discrete representations with vector quantization (VQ) has emerged as a powerful approach in various generative models. However, most VQ-based models rely on a single, fixed-rate codebook, requiring extensive retraining for new bitrates or efficiency requirements. We introduce Rate-Adaptive Quantization (RAQ), a multi-rate codebook adaptation framework for VQ-based generative models. RAQ applies a data-driven approach to generate variable-rate codebooks from a single baseline VQ model, enabling flexible tradeoffs between compression and reconstruction fidelity. Additionally, we provide a simple clustering-based procedure for pre-trained VQ models, offering an alternative when retraining is infeasible. Our experiments show that RAQ performs effectively across multiple rates, often outperforming conventional fixed-rate VQ baselines. By enabling a single system to seamlessly handle diverse bitrate requirements, RAQ extends the adaptability of VQ-based generative models and broadens their applicability to data compression, reconstruction, and generation tasks.