NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models
This research is significant for developers and evaluators of foundation models, as it highlights a critical issue where models might be recalling memorized data rather than demonstrating true out-of-sample skill, potentially leading to overestimation of model capabilities.
The paper introduces NumLeak, a framework to measure the memorization of public numeric benchmarks by foundation models. It found that top-tier LLMs recall financial and economic data with high fidelity (Pearson r=0.97-0.99) and that this recall persists even on recent holdout data, with a significant drop in parse rate but not accuracy for answered months. The study also demonstrated that a Sonnet model's market-sentiment regression, initially correlating at r=0.74 with true market returns, collapses to r=0.02 after residualizing the model's own recall.
Public numeric benchmarks appear in pretraining, so an evaluation that conditions on a date may be measuring memorized recall rather than out-of-sample skill. We introduce NumLeak, a measurement framework that combines API-boundary probes on production models with a white-box controlled validation on an open causal LM. Top-tier frontier LLMs recall the Fama-French market excess return at 3-seed pooled Pearson r=0.97-0.99 while staying within 0.15 within-25bps on the five sibling factors; comparable fidelity appears on U.S. unemployment, CPI inflation, and NOAA temperature. On a recent-release holdout, parse rate collapses to 21-57% but r stays at approximately 0.99 on months answered, the refuse-or-recall asymmetry a memorized channel predicts. The white-box experiment reproduces the dose-response, and logprob ranking detects memorization that open-ended generation misses, implying closed-API black-box probes understate the channel. A Sonnet "date to market-sentiment" regression that correlates with true Mkt-RF at r=0.74 collapses to r=0.02 once the model's own recall is residualized out. A one-line system-prompt defense blocks 99.8% of a non-adaptive single-turn suffix attack set at near-zero utility cost on conceptual and historical-narrative queries