Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies
This paper provides a systematic security analysis for researchers and developers of self-evolving LLM agent systems, revealing that static defenses are structurally inadequate against these new threats.
Self-evolving LLM agent systems introduce a new threat landscape where adversarial influences become permanently encoded and self-amplify. The study identifies 17 out of 25 attack surface cells with critical threats, and shows that evolution-native design activates 3.5× more attack surface cells with 100% attack persistence, while co-located security scanners block only 2.5% of attacks.
Self-evolving LLM agent systems, which autonomously update their model parameters, memory, tools, and architectures, introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded, self-amplify across generations, and propagate through populations without sustained attacker access. We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface (MLAS) matrix, which decomposes the attack surface into five functional modules (Brain, Cognitive Resource, Execution, Self-Design, Collective) $\times$ five lifecycle stages (Bootstrap, Propose, Evaluate, Commit, Serve). Analysis of the resulting 25 cells reveals that 17 face critical threats for which no effective partial mitigation. We identify seven cross-cutting amplification effects that interact synergistically and cannot be addressed by securing individual modules in isolation. Comparative case studies of two open-source frameworks demonstrate that evolution-native design activates $3.5\times$ more attack surface cells and achieves a 100% attack persistence rate (40/40 payloads across all CIA+Privacy categories), while co-located security scanners block only 2.5% of attacks. Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent, gives rise to entirely new attack classes, and renders static defenses structurally inadequate, motivating evolution-aware security frameworks and formal verification for self-modifying systems.