When Reranking Hurts: Uncertainty-Based Gating for Few-Shot Reranking
For practitioners using few-shot learning with LLMs, this work provides a method to avoid performance degradation and reduce costs by selectively applying reranking.
The paper challenges the assumption that reranking always improves few-shot selection, showing it can degrade performance. They propose Training-Free Gated Reranking, which reduces computational costs by 15%-80% while improving average performance by up to 2% across multiple LLMs and tasks.
Few-shot selection typically assumes that reranking retrieved examples always improves performance. We challenge this view by identifying that the expensive reranking step can in fact degrade performance. Instead, we propose \emph{Training-Free Gated Reranking}, which decides whether to rerank the few-shot examples based on the model's uncertainty. Extensive experiments across 8 LLMs, covering 7 NLU datasets and 9 MT domain-language combinations, demonstrate that our approach reduces computational costs by 15\%-80\% while improving average performance by up to 2\%. These findings indicate that higher computational cost does not guarantee better performance, and that reranking is most beneficial when targeted at high-uncertainty instances.