Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support
This work addresses the need for human-aligned, actionable evaluation to improve therapeutic quality in mental health LLMs, offering a framework that significantly enhances response safety and effectiveness.
The paper introduces TheraJudge, a therapeutic evaluator trained via preference-based optimization, and TheraAgent, a multi-agent system that uses TheraJudge's evaluations to refine therapeutic responses. TheraAgent achieves a +0.43 improvement in human-rated therapeutic quality on a 5-point scale, with 96% clinician inter-rater reliability, and low-quality responses improve by +2.45 points with a 94% recovery rate.
Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric. We introduce a framework that formulates therapeutic response generation as a decision-refinement problem driven by multi-dimensional, human-aligned evaluation. In Stage I, we introduce TheraJudge, an open-source therapeutic evaluator trained via preference-based optimization on human-annotated data to produce reliable judgments across 7 psychological dimensions. In Stage II, we introduce TheraAgent, which operationalizes TheraJudge's evaluations through a coordinated refinement process with specialized Critic, Coach, and Therapist roles that translate evaluative signals into targeted response revisions. Empirically, TheraJudge achieves strong agreement with clinician ratings, with intraclass correlation coefficients (ICC = 0.87-0.95), surpassing supervised baselines and strong closed-source judges, particularly on critical dimensions such as Safety, Relevance, and Empathy. Acting on these evaluations, TheraAgent yields a +0.43 improvement in human-rated therapeutic quality (on a 5-point scale) under blind evaluation, with 96\% clinician inter-rater reliability. Low-quality responses ($\leq 3$) improve by +2.45 points with a 94\% recovery rate, demonstrating targeted correction of unsafe outputs. Overall, our results indicate that effective alignment of mental-health LLMs stems from acting on human-aligned evaluation, rather than relying solely on stronger generation. We release code at https://github.com/vis-nlp/TheraAlign.