On the Impossibility of Separating Intelligence from Judgment: The Computational Intractability of Filtering for AI Alignment
This work addresses the AI alignment challenge for developers and policymakers, revealing fundamental computational barriers to external safety filters, indicating it is incremental in formalizing known concerns.
The paper tackles the problem of aligning large language models (LLMs) by filtering harmful content, showing that both input and output filtering are computationally intractable under cryptographic assumptions, with adversarial prompts being indistinguishable from benign ones and output filtering proven intractable in natural settings.
With the increased deployment of large language models (LLMs), one concern is their potential misuse for generating harmful content. Our work studies the alignment challenge, with a focus on filters to prevent the generation of unsafe information. Two natural points of intervention are the filtering of the input prompt before it reaches the model, and filtering the output after generation. Our main results demonstrate computational challenges in filtering both prompts and outputs. First, we show that there exist LLMs for which there are no efficient prompt filters: adversarial prompts that elicit harmful behavior can be easily constructed, which are computationally indistinguishable from benign prompts for any efficient filter. Our second main result identifies a natural setting in which output filtering is computationally intractable. All of our separation results are under cryptographic hardness assumptions. In addition to these core findings, we also formalize and study relaxed mitigation approaches, demonstrating further computational barriers. We conclude that safety cannot be achieved by designing filters external to the LLM internals (architecture and weights); in particular, black-box access to the LLM will not suffice. Based on our technical results, we argue that an aligned AI system's intelligence cannot be separated from its judgment.