Embedding Compression via Spherical Coordinates
This provides a more efficient compression technique for embeddings used in text, image, and multi-vector applications, though it is incremental as it builds on existing lossless methods.
The paper tackles the problem of compressing unit-norm embeddings by introducing a method that exploits the concentration of spherical coordinates around π/2, achieving 1.5× compression with reconstruction error below 1e-7, which is 25% better than prior lossless methods.
We present a compression method for unit-norm embeddings that achieves 1.5$\times$ compression, 25% better than the best prior lossless method. The method exploits that spherical coordinates of high-dimensional unit vectors concentrate around $π/2$, causing IEEE 754 exponents to collapse to a single value and high-order mantissa bits to become predictable, enabling entropy coding of both. Reconstruction error is below 1e-7, under float32 machine epsilon. Evaluation across 26 configurations spanning text, image, and multi-vector embeddings confirms consistent improvement.