Invasive Context Engineering to Control Large Language Models
This work addresses security vulnerabilities in LLMs for users requiring robust guarantees, but it appears incremental as it builds on existing control methods like prompting.
The paper tackles the problem of controlling Large Language Models (LLMs) to prevent abuse and jailbreaks, especially in long-context scenarios, by proposing invasive context engineering with inserted control sentences, which avoids training and addresses data shortages.
Current research on operator control of Large Language Models improves model robustness against adversarial attacks and misbehavior by training on preference examples, prompting, and input/output filtering. Despite good results, LLMs remain susceptible to abuse, and jailbreak probability increases with context length. There is a need for robust LLM security guarantees in long-context situations. We propose control sentences inserted into the LLM context as invasive context engineering to partially solve the problem. We suggest this technique can be generalized to the Chain-of-Thought process to prevent scheming. Invasive Context Engineering does not rely on LLM training, avoiding data shortage pitfalls which arise in training models for long context situations.