The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliability
This paper redefines the problem of LLM reliability for practitioners deploying LLMs in specific, bounded domains, suggesting that achieving reliability is feasible within these contexts by focusing on local failure catalogues. It offers an incremental shift in perspective for LLM developers and researchers.
The paper argues that universal LLM reliability is impossible due to an unbounded number of failure modes. However, within operationally bounded "patches" (e.g., legal review, medical RAG), failures are sparse and repetitive, making reliability a local problem of discovering and covering these recurring failure modes. The authors formalize this with propositions showing that while no finite intervention dictionary covers all modes in an unbounded domain, a sufficient per-hard-decision intervention budget grows polylogarithmically in sequence length and becomes domain-constant once the patch catalogue saturates.
Universal LLM reliability is not a finite-library problem: across all possible tasks, tools, schemas, knowledge sources, and evaluator expectations, new intervention-distinguishable failure modes can appear without bound, so no finite intervention dictionary can guarantee bounded residual error for every such mode. But deployed systems do not operate over the whole universe. They operate inside operationally bounded patches (legal review, medical RAG, code repair, customer-support agents, contract extraction) with recurring tasks, schemas, tools, and evaluator expectations. Within such patches, empirical evidence suggests failures are sparse, repetitive, and concentrated in a small recurring catalogue, so reliability becomes a local catalogue-discovery and intervention-coverage problem rather than an exponential token-length problem. We formalize this transition with two propositions and one corollary. Proposition 1 is the worst-case-mode-wise negative result: no finite intervention dictionary covers every distinguishable failure mode of an unbounded domain. Corollary 1 is the inverse-discovery implication: the logarithmic upper bound on mode discovery cannot accommodate linearly more distinct tail modes without exponentially more observed hard-failure events. Proposition 2 is the positive patch-local result: under log active-mode exposure and head-heavy coverage, a sufficient per-hard-decision intervention budget grows polylogarithmically in sequence length and becomes domain-constant once the patch catalogue saturates. The framework relocates rather than dissolves long-context difficulty: where the number of hard decisions itself grows with task length, reliability remains hard; the contribution is to identify the on-axis intervention rather than to make those regimes easy.