Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization
This work addresses the need for adaptive communication bottlenecks in multi-agent systems and other applications, representing an incremental improvement over existing VQ-based methods.
The paper tackles the problem of fixed discretization tightness in vector quantization methods by proposing dynamic selection of tightness conditioned on inputs, showing that this improves model performance on visual reasoning and reinforcement learning tasks.
Vector Quantization (VQ) is a method for discretizing latent representations and has become a major part of the deep learning toolkit. It has been theoretically and empirically shown that discretization of representations leads to improved generalization, including in reinforcement learning where discretization can be used to bottleneck multi-agent communication to promote agent specialization and robustness. The discretization tightness of most VQ-based methods is defined by the number of discrete codes in the representation vector and the codebook size, which are fixed as hyperparameters. In this work, we propose learning to dynamically select discretization tightness conditioned on inputs, based on the hypothesis that data naturally contains variations in complexity that call for different levels of representational coarseness. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.