An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc
This work addresses a production bottleneck for deploying sparse retrieval models in search engines, offering a practical solution with incremental improvements to existing regularization methods.
The paper tackles the problem of high retrieval latency in Learned Sparse Retrieval models like SPLADE, which arises from terms with high document frequencies, by introducing DF-FLOPS regularization that penalizes such terms. The result is a 10x faster retrieval latency while maintaining effectiveness, with only a 2.2-point drop in MRR@10 in-domain and improved cross-domain performance in most tasks.
Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms - only within a given query or document. Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists. To address the issue of high DFs, we present a new variant of FLOPS regularization: DF-FLOPS. This new regularization technique penalizes the usage of high-DF terms, thereby shortening posting lists and reducing retrieval latency. Unlike other inference-time sparsification methods, such as stopword removal, DF-FLOPS regularization allows for the selective inclusion of high-frequency terms in cases where the terms are truly salient. We find that DF-FLOPS successfully reduces the prevalence of high-DF terms and lowers retrieval latency (around 10x faster) in a production-grade engine while maintaining effectiveness both in-domain (only a 2.2-point drop in MRR@10) and cross-domain (improved performance in 12 out of 13 tasks on which we tested). With retrieval latencies on par with BM25, this work provides an important step towards making LSR practical for deployment in production-grade search engines.