Can LLMs Be Constrained to the Past? Improving Knowledge Cutoff through Recall-Based Prompting
For practitioners needing temporally constrained LLMs (e.g., historical QA, compliance), this work provides simple prompting methods that improve cutoff adherence, though the gains are incremental over existing prompting baselines.
The paper tackles the problem of enforcing knowledge cutoffs in LLMs to prevent use of post-cutoff information. The proposed recall-based prompting strategies (Self-Recall and Question-Recall) outperform direct-answer and reasoning baselines, achieving particularly strong gains on counterfactual questions, with combined SR+QR yielding the best performance across multiple cutoff years.
Prompted knowledge cutoff instructs a large language model (LLM) to act as if information beyond a specified cutoff date were unavailable. However, prior work mainly relies on direct-answer generation, which struggles when post-cutoff knowledge is not explicitly queried but is only causally related to the question. To address this limitation, we propose two recall-based prompting strategies: Self-Recall (SR), which asks the model to restate its cutoff constraint, and Question-Recall (QR), which requires the model to recall question-relevant information valid under the cutoff. Across three existing benchmarks, our methods outperform both direct-answer prompting and conventional step-by-step reasoning baselines, with particularly strong improvements on counterfactual questions. To investigate robustness across different cutoff settings, we further construct the Multi-cutoff Historical Event Benchmark (MHEB), which evaluates the same question under multiple cutoff years. Results show that knowledge cutoff performance varies with cutoff distance, while combining SR and QR consistently yields the best performance.