Interpretable Risk Mitigation in LLM Agent Systems
This addresses safety concerns for LLM agent reliability in domains requiring responsible action, though it is incremental as it builds on existing representation-steering techniques.
The paper tackles the problem of unpredictable behavior in LLM-powered autonomous agents by introducing a strategy-modification method that steers the residual stream using interpretable features from a sparse autoencoder, reducing the average defection probability by 28 percentage points in a game-theoretic environment.
Autonomous agents powered by large language models (LLMs) enable novel use cases in domains where responsible action is increasingly important. Yet the inherent unpredictability of LLMs raises safety concerns about agent reliability. In this work, we explore agent behaviour in a toy, game-theoretic environment based on a variation of the Iterated Prisoner's Dilemma. We introduce a strategy-modification method-independent of both the game and the prompt-by steering the residual stream with interpretable features extracted from a sparse autoencoder latent space. Steering with the good-faith negotiation feature lowers the average defection probability by 28 percentage points. We also identify feasible steering ranges for several open-source LLM agents. Finally, we hypothesise that game-theoretic evaluation of LLM agents, combined with representation-steering alignment, can generalise to real-world applications on end-user devices and embodied platforms.