On The Effectiveness-Fluency Trade-Off In LLM Conditioning: A Systematic Study
For practitioners deploying LLMs, this study reveals critical trade-offs and interactions between conditioning methods and model training paradigms, guiding method selection.
The paper systematically evaluates LLM conditioning methods, finding that efficient steering methods often degrade fluency, and activation steering is less effective on instruction-tuned models. Prompting and fine-tuning work for concept injection but not removal.
Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet a clear understanding of the involved trade-offs remains elusive. Current approaches to conditioning are often evaluated with a narrow focus on their effectiveness at injecting or removing a target concept, neglecting generation quality. We systematically investigate a range of conditioning methods in both injection and removal scenarios. We find that efficient steering methods frequently achieve conditioning at a steep cost to fluency. Furthermore, we identify a critical yet previously overlooked interaction with the training paradigm: activation steering methods are far less effective on instruction-tuned models than on their base counterparts. Simple prompting and full-fledged supervised fine-tuning, on the other hand, are viable options for concept injection, but are not as good at concept removal. Finally, cheaply computed textual metrics highly correlate to costly LLM-as-judge scores, and provide insights on the behavior of conditioning methods.