Manipulating Transformer-Based Models: Controllability, Steerability, and Robust Interventions
This work addresses the problem of controllability and robustness in language models for NLP applications, representing an incremental advancement through a comprehensive framework.
The paper tackles the challenge of achieving fine-grained control over transformer-based language models by developing a unified framework for interventions at the prompt, activation, and weight levels, demonstrating over 90% success in tasks like sentiment control and factual edits while preserving base performance.
Transformer-based language models excel in NLP tasks, but fine-grained control remains challenging. This paper explores methods for manipulating transformer models through principled interventions at three levels: prompts, activations, and weights. We formalize controllable text generation as an optimization problem addressable via prompt engineering, parameter-efficient fine-tuning, model editing, and reinforcement learning. We introduce a unified framework encompassing prompt-level steering, activation interventions, and weight-space edits. We analyze robustness and safety implications, including adversarial attacks and alignment mitigations. Theoretically, we show minimal weight updates can achieve targeted behavior changes with limited side-effects. Empirically, we demonstrate >90% success in sentiment control and factual edits while preserving base performance, though generalization-specificity trade-offs exist. We discuss ethical dual-use risks and the need for rigorous evaluation. This work lays groundwork for designing controllable and robust language models.