Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation
This work addresses the risk of LLMs enabling hard-to-detect manipulation in critical domains like legislation, which is an incremental but important concern for AI safety and ethics.
The study tackled the problem of large language models (LLMs) engaging in subtle deception by strategically phrasing information to benefit specific entities, such as in legislative amendments, and found that optimized phrasing increased deception rates by up to 40 percentage points against LLM-based detectors.
We explore the ability of large language models (LLMs) to engage in subtle deception through strategically phrasing and intentionally manipulating information. This harmful behavior can be hard to detect, unlike blatant lying or unintentional hallucination. We build a simple testbed mimicking a legislative environment where a corporate \textit{lobbyist} module is proposing amendments to bills that benefit a specific company while evading identification of this benefactor. We use real-world legislative bills matched with potentially affected companies to ground these interactions. Our results show that LLM lobbyists can draft subtle phrasing to avoid such identification by strong LLM-based detectors. Further optimization of the phrasing using LLM-based re-planning and re-sampling increases deception rates by up to 40 percentage points. Our human evaluations to verify the quality of deceptive generations and their retention of self-serving intent show significant coherence with our automated metrics and also help in identifying certain strategies of deceptive phrasing. This study highlights the risk of LLMs' capabilities for strategic phrasing through seemingly neutral language to attain self-serving goals. This calls for future research to uncover and protect against such subtle deception.