GPT-Red: Automated Red Teaming via Self-Play at Scale
For AI safety researchers, this work provides a scalable method to automatically discover and defend against prompt injection attacks, significantly improving robustness of production LLMs.
GPT-Red is an automated red-teaming agent trained via self-play to discover novel prompt injection attacks against frontier LLMs. It successfully breaks past models up to GPT-5.5, finds more attacks than human red-teamers, and generalizes to held-out environments, enabling adversarial training of GPT-5.6, the most robust model to prompt injections to date.
We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for \textit{even stronger} red-teamer agents, thus unlocking a self-improvement flywheel.