Epistemic Compression: The Case for Deliberate Ignorance in High-Stakes AI
This addresses reliability problems in high-stakes AI applications, proposing a shift from scaling to principled parsimony, though it appears incremental as it builds on regularization concepts.
The paper tackles the Fidelity Paradox, where foundation models fail in high-stakes domains like medicine and finance due to structural issues, by introducing Epistemic Compression, a principle that matches model complexity to data shelf life, achieving 86.7% concordance in empirical tests across 15 domains.
Foundation models excel in stable environments, yet often fail where reliability matters most: medicine, finance, and policy. This Fidelity Paradox is not just a data problem; it is structural. In domains where rules change over time, extra model capacity amplifies noise rather than capturing signal. We introduce Epistemic Compression: the principle that robustness emerges from matching model complexity to the shelf life of the data, not from scaling parameters. Unlike classical regularization, which penalizes weights post hoc, Epistemic Compression enforces parsimony through architecture: the model structure itself is designed to reduce overfitting by making it architecturally costly to represent variance that exceeds the evidence in the data. We operationalize this with a Regime Index that separates Shifting Regime (unstable, data-poor; simplicity wins) from Stable Regime (invariant, data-rich; complexity viable). In an exploratory synthesis of 15 high-stakes domains, this index was concordant with the empirically superior modeling strategy in 86.7% of cases (13/15). High-stakes AI demands a shift from scaling for its own sake to principled parsimony.