DeCAL Tokenwise Compression
This addresses the problem of efficient storage and retrieval of dense representations for NLP practitioners, offering significant savings where pre-computed embeddings are used, though it appears incremental in its modifications.
The paper tackles tokenwise compression by introducing DeCAL, which uses a modified encoder-decoder language model to produce high-quality compressed representations, achieving performance matching uncompressed models at 2x compression with only minor dropoffs up to 8x compression on tasks like question-answering and summarization.
This paper introduces DeCAL, a new method for tokenwise compression. DeCAL uses an encoder-decoder language model pretrained with denoising to learn to produce high-quality, general-purpose compressed representations from the encoder. DeCAL applies small modifications to the encoder, with the emphasis on maximizing compression quality, even at the expense of compute. We show that DeCAL at 2x compression can match uncompressed on several downstream tasks, with usually only a minor dropoff in metrics up to 8x compression, among question-answering, summarization, and multi-vector retrieval tasks. DeCAL offers significant savings where pre-computed dense representations can be utilized, and we believe the approach can be further developed to be more broadly applicable.