Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
For researchers and practitioners of LLM mathematical reasoning, this provides a fine-grained method to detect and correct reasoning failures at the token level, with demonstrated improvements.
The paper identifies 'cliff tokens'—single tokens where LLM reasoning shifts from correct to incorrect—using a statistical test on token-wise potential. Deleting the first cliff token and resampling recovers pass@64 to 1.0, and a preference optimization method (Cliff-DPO) improves accuracy by up to +6.6 across benchmarks.
Large language models (LLMs) reach high accuracy in mathematical reasoning, but individual traces on the same problem diverge; some arrive at the correct answer while others fail. Prior work analyzes failure at the step, chunk, or sentence level, or at tokens where failure has already occurred. Neither identifies the precise token that triggers the shift toward failure. We introduce the cliff token, a token where the token-wise potential drops significantly under an adaptive threshold that scales with the local token-wise potential, based on a one-sided two-proportion z-test. Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to between 0.71 and 1.00. We further introduce a cliff taxonomy of deterministic, uncertain, and sampled-off cliffs, defined by greedy choice and token entropy. Each type has distinct probabilistic characteristics, and the taxonomy generalizes across model scales. Finally, we validate the taxonomy via single-token preference optimization at cliff positions (Cliff-DPO). Trained on GSM8K, Cliff-DPO improves accuracy across benchmarks by up to +6.6. Optimizing at uncertain and sampled-off cliffs improves reasoning, while deterministic cliffs do not.