Ada-RS: Adaptive Rejection Sampling for Selective Thinking
This work addresses efficiency for latency-sensitive deployments of tool-using LLMs, representing an incremental improvement in training-signal selection.
The paper tackled the problem of inefficient token usage in chain-of-thought reasoning for large language models in cost-sensitive settings by introducing Ada-RS, an adaptive rejection sampling framework, which reduced average output tokens by up to 80% and thinking rate by up to 95% while maintaining tool call accuracy on a synthetic e-commerce benchmark.
Large language models (LLMs) are increasingly being deployed in cost and latency-sensitive settings. While chain-of-thought improves reasoning, it can waste tokens on simple requests. We study selective thinking for tool-using LLMs and introduce Adaptive Rejection Sampling (Ada-RS), an algorithm-agnostic sample filtering framework for learning selective and efficient reasoning. For each given context, Ada-RS scores multiple sampled completions with an adaptive length-penalized reward then applies stochastic rejection sampling to retain only high-reward candidates (or preference pairs) for downstream optimization. We demonstrate how Ada-RS plugs into both preference pair (e.g. DPO) or grouped policy optimization strategies (e.g. DAPO). Using Qwen3-8B with LoRA on a synthetic tool call-oriented e-commerce benchmark, Ada-RS improves the accuracy-efficiency frontier over standard algorithms by reducing average output tokens by up to 80% and reducing thinking rate by up to 95% while maintaining or improving tool call accuracy. These results highlight that training-signal selection is a powerful lever for efficient reasoning in latency-sensitive deployments.