Navigating through the hidden embedding space: steering LLMs to improve mental health assessment
This work addresses the challenge of domain-specific adaptation for LLMs in mental health applications, offering a cost-efficient method, though it is incremental in nature.
The study tackled the problem of improving mental health assessment capabilities of large language models (LLMs) by applying a lightweight linear transformation to activations, achieving improved results in relevance prediction and questionnaire completion tasks on Reddit data.
The rapid evolution of Large Language Models (LLMs) is transforming AI, opening new opportunities in sensitive and high-impact areas such as Mental Health (MH). Yet, despite these advancements, recent evidence reveals that smaller-scale models still struggle to deliver optimal performance in domain-specific applications. In this study, we present a cost-efficient yet powerful approach to improve MH assessment capabilities of an LLM, without relying on any computationally intensive techniques. Our lightweight method consists of a linear transformation applied to a specific layer's activations, leveraging steering vectors to guide the model's output. Remarkably, this intervention enables the model to achieve improved results across two distinct tasks: (1) identifying whether a Reddit post is useful for detecting the presence or absence of depressive symptoms (relevance prediction task), and (2) completing a standardized psychological screening questionnaire for depression based on users' Reddit post history (questionnaire completion task). Results highlight the untapped potential of steering mechanisms as computationally efficient tools for LLMs' MH domain adaptation.