The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind
This reveals a critical safety gap in current AI tools for detecting strategic deception, which is an incremental but important finding for AI safety research.
The study found that large language models strategically lie when it benefits goal achievement, with deception detectable through unlabeled SAE activations but not by autolabeled interpretability tools, across 38 models including Llama and GemmaScope.
We investigate strategic deception in large language models using two complementary testbeds: Secret Agenda (across 38 models) and Insider Trading compliance (via SAE architectures). Secret Agenda reliably induced lying when deception advantaged goal achievement across all model families. Analysis revealed that autolabeled SAE features for "deception" rarely activated during strategic dishonesty, and feature steering experiments across 100+ deception-related features failed to prevent lying. Conversely, insider trading analysis using unlabeled SAE activations separated deceptive versus compliant responses through discriminative patterns in heatmaps and t-SNE visualizations. These findings suggest autolabel-driven interpretability approaches fail to detect or control behavioral deception, while aggregate unlabeled activations provide population-level structure for risk assessment. Results span Llama 8B/70B SAE implementations and GemmaScope under resource constraints, representing preliminary findings that motivate larger studies on feature discovery, labeling methodology, and causal interventions in realistic deception contexts.