Representation Engineering for Large-Language Models: Survey and Research Challenges
This work addresses the unpredictability of LLMs for developers and users, but it is incremental as it builds on existing interpretability and control methods.
The paper tackles the problem of making large-language models more predictable and tractable by introducing representation engineering, which uses contrasting inputs to detect and edit high-level concept representations like honesty or harmfulness. It formalizes this approach, compares it to alternatives like mechanistic interpretability, and outlines risks and future research directions.
Large-language models are capable of completing a variety of tasks, but remain unpredictable and intractable. Representation engineering seeks to resolve this problem through a new approach utilizing samples of contrasting inputs to detect and edit high-level representations of concepts such as honesty, harmfulness or power-seeking. We formalize the goals and methods of representation engineering to present a cohesive picture of work in this emerging discipline. We compare it with alternative approaches, such as mechanistic interpretability, prompt-engineering and fine-tuning. We outline risks such as performance decrease, compute time increases and steerability issues. We present a clear agenda for future research to build predictable, dynamic, safe and personalizable LLMs.