LLM Reinforcement in Context
This addresses alignment scaling for LLM users, but appears incremental as it builds on existing methods.
The paper tackles the problem of LLM alignment weakening with longer user inputs by proposing interruptions, control sentences inserted periodically, and reports that this approach can be generalized to Chain-of-Thought to prevent scheming.
Current Large Language Model alignment research mostly focuses on improving model robustness against adversarial attacks and misbehavior by training on examples and prompting. Research has shown that LLM jailbreak probability increases with the size of the user input or conversation length. There is a lack of appropriate research into means of strengthening alignment which also scale with user input length. We propose interruptions as a possible solution to this problem. Interruptions are control sentences added to the user input approximately every x tokens for some arbitrary x. We suggest that this can be generalized to the Chain-of-Thought process to prevent scheming.