Unvalidated Trust: Cross-Stage Vulnerabilities in Large Language Model Architectures
This addresses security vulnerabilities in multi-stage LLM pipelines for developers and users, though it is incremental as it builds on existing zero-trust concepts.
The paper identifies 41 recurring risk patterns in commercial LLM architectures due to unvalidated trust between processing stages, showing that inputs can trigger unintended responses or state changes, and proposes zero-trust principles like provenance enforcement and a 'Countermind' blueprint for mitigation.
As Large Language Models (LLMs) are increasingly integrated into automated, multi-stage pipelines, risk patterns that arise from unvalidated trust between processing stages become a practical concern. This paper presents a mechanism-centered taxonomy of 41 recurring risk patterns in commercial LLMs. The analysis shows that inputs are often interpreted non-neutrally and can trigger implementation-shaped responses or unintended state changes even without explicit commands. We argue that these behaviors constitute architectural failure modes and that string-level filtering alone is insufficient. To mitigate such cross-stage vulnerabilities, we recommend zero-trust architectural principles, including provenance enforcement, context sealing, and plan revalidation, and we introduce "Countermind" as a conceptual blueprint for implementing these defenses.