Test-Time Compute for Dense Retrieval: Agentic Program Generation with Frozen Embedding Models
For practitioners using dense retrieval, this provides a simple, parameter-free method to improve retrieval accuracy without retraining or changing the embedding model.
The paper shows that test-time compute benefits small frozen embedding models, not just large reasoning models. An agentic program-search loop discovered a softmax-weighted centroid interpolation that lifts nDCG@10 significantly across seven embedding model families.
Test-time compute is widely believed to benefit only large reasoning models. We show it also helps small embedding models. Most modern embedding checkpoints are distilled from large LLM backbones and inherit their representation space; a frozen embedding model should therefore benefit from extra inference compute without retraining. Using an agentic program-search loop, we explore 259 candidate inference programs over a frozen embedding API across ninety generations. The entire Pareto frontier collapses onto a single algebra: a softmax-weighted centroid of the local top-K documents interpolated with the query. This parameter-free default lifts nDCG@10 statistically significantly across seven embedding-model families spanning a tenfold parameter range, with held-out full-BEIR validation confirming the lift on every model tested.