Steerable Chatbots: Personalizing LLMs with Preference-Based Activation Steering
This addresses the challenge for lay users who struggle to specify preferences to AI assistants, offering a lightweight personalization method, though it is incremental as it builds on existing activation steering techniques.
The paper tackled the problem of personalizing LLM responses to align with user preferences without requiring extensive user history, by using activation steering during inference, and found it effectively aligned conversations with hidden preferences in a user study with 14 participants.
As large language models (LLMs) improve in their capacity to serve as personal AI assistants, their ability to output uniquely tailored, personalized responses that align with the soft preferences of their users is essential for enhancing user satisfaction and retention. However, untrained lay users have poor prompt specification abilities and often struggle with conveying their latent preferences to AI assistants. To address this, we leverage activation steering to guide LLMs to align with interpretable preference dimensions during inference. In contrast to memory-based personalization methods that require longer user history, steering is extremely lightweight and can be easily controlled by the user via an linear strength factor. We embed steering into three different interactive chatbot interfaces and conduct a within-subjects user study (n=14) to investigate how end users prefer to personalize their conversations. The results demonstrate the effectiveness of preference-based steering for aligning real-world conversations with hidden user preferences, and highlight further insights on how diverse values around control, usability, and transparency lead users to prefer different interfaces.