Kernel $k$-Medoids as General Vector Quantization
This work provides theoretical insight into unifying distance-based and density-based vector quantization methods, which is incremental for researchers in machine learning and data compression.
The paper tackles the connection between k-medoids clustering and Kernel Density Estimation in vector quantization by showing that a KDE-based QUBO formulation is a special case of a k-medoids QUBO under mild kernel assumptions, revealing a deeper structural relationship and geometric insights into weighting parameters.
Vector Quantization (VQ) is a widely used technique in machine learning and data compression, valued for its simplicity and interpretability. Among hard VQ methods, $k$-medoids clustering and Kernel Density Estimation (KDE) approaches represent two prominent yet seemingly unrelated paradigms -- one distance-based, the other rooted in probability density matching. In this paper, we investigate their connection through the lens of Quadratic Unconstrained Binary Optimization (QUBO). We compare a heuristic QUBO formulation for $k$-medoids, which balances centrality and diversity, with a principled QUBO derived from minimizing Maximum Mean Discrepancy in KDE-based VQ. Surprisingly, we show that the KDE-QUBO is a special case of the $k$-medoids-QUBO under mild assumptions on the kernel's feature map. This reveals a deeper structural relationship between these two approaches and provides new insight into the geometric interpretation of the weighting parameters used in QUBO formulations for VQ.