Bang for the Buck: Vector Search on Cloud CPUs
This work helps users optimize cost-performance for deploying vector search systems in the cloud, though it is incremental as it benchmarks existing hardware without introducing new methods.
The study tackled the problem of selecting cloud CPUs for vector search by benchmarking performance across different microarchitectures, finding that AMD's Zen4 offers nearly 3x higher queries per second than Intel's Sapphire Rapids for IVF indexes, but Graviton3 provides the best queries per dollar for most settings.
Vector databases have emerged as a new type of systems that support efficient querying of high-dimensional vectors. Many of these offer their database as a service in the cloud. However, the variety of available CPUs and the lack of vector search benchmarks across CPUs make it difficult for users to choose one. In this study, we show that CPU microarchitectures available in the cloud perform significantly differently across vector search scenarios. For instance, in an IVF index on float32 vectors, AMD's Zen4 gives almost 3x more queries per second (QPS) compared to Intel's Sapphire Rapids, but for HNSW indexes, the tables turn. However, when looking at the number of queries per dollar (QP$), Graviton3 is the best option for most indexes and quantization settings, even over Graviton4 (Table 1). With this work, we hope to guide users in getting the best "bang for the buck" when deploying vector search systems.