MSRS: Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models
This addresses the challenge of precise multi-attribute control in LLMs for applications like content moderation or style adaptation, though it is an incremental improvement over existing activation steering methods.
The paper tackled the problem of jointly steering multiple attributes in Large Language Models without interference, and the result was that MSRS significantly reduces attribute conflicts and surpasses existing methods across various attributes.
Activation steering offers a promising approach to controlling the behavior of Large Language Models by directly manipulating their internal activations. However, most existing methods struggle to jointly steer multiple attributes, often resulting in interference and undesirable trade-offs. To address this challenge, we propose Multi-Subspace Representation Steering (MSRS), a novel framework for effective multi-attribute steering via subspace representation fine-tuning. MSRS reduces inter-attribute interference by allocating orthogonal subspaces to each attribute, isolating their influence within the model's representation space. MSRS also incorporates a hybrid subspace composition strategy: it combines attribute-specific subspaces for unique steering directions with a shared subspace for common steering directions. A dynamic weighting function learns to efficiently integrate these components for precise control. During inference, MSRS introduces a token-level steering mechanism that dynamically identifies and intervenes on the most semantically relevant tokens, enabling fine-grained behavioral modulation. Experimental results show that MSRS significantly reduces attribute conflicts, surpasses existing methods across a range of attributes, and generalizes effectively to diverse downstream tasks.