Summoning the Oracle to Slay It: Mitigating Look-Ahead Bias in Financial Backtesting with Large Language Models
For practitioners using LLMs for financial backtesting, this work mitigates a critical bias that invalidates historical performance evaluation, though the method is domain-specific and incremental.
The paper identifies parametric look-ahead bias in LLM-based financial backtesting, where models trained after historical events inadvertently memorize outcomes. It proposes FinCAD, an inference-time adaptation that suppresses this memory, reducing in-sample backtest returns by up to 67.1% while keeping out-of-sample returns within $8K and Sharpe within 0.10 of baseline, and improving ranking correlation from 0.779 to 0.846.
Backtesting large language models (LLMs) on historical financial data is unreliable because pre-training cuts off after the events happened. An LLM trained in 2024 already "knows" which way 2018-2020 stocks moved. We name this failure parametric look-ahead bias and propose FinCAD, an inference-time adaptation of Context-Aware Decoding that suppresses an LLM's memory of historical outcomes without retraining. FinCAD pairs an adversarial bias-discovery pipeline that learns a model-specific memory-activating prior prompt with an entity- and date-adaptive rule that scales the CAD strength to per-(entity, date) memorisation, so the penalty fires on memorised in-sample dates and decays to zero out-of-sample. Across five 7-14B LLMs and five mega-cap equities, FinCAD cuts in-sample backtest returns by up to -67.1% on memorised dates while leaving 2025 out-of-sample returns within $8K and Sharpe within 0.10 of baseline, and preserves general-purpose reasoning within 1.7 pts. On an eleven-model leaderboard, it raises the in-sample / out-of-sample Spearman correlation from +0.779 to +0.846, recovering rankings that genuinely predict out-of-sample performance.