Assessing the Effectiveness of LLMs in Delivering Cognitive Behavioral Therapy
This addresses the problem of scalable mental health solutions for individuals seeking accessible therapy, but it is incremental as it assesses existing methods on new data without major breakthroughs.
The paper evaluated the ability of Large Language Models (LLMs) to emulate professional therapists in Cognitive Behavioral Therapy (CBT), finding that while LLMs can generate CBT-like dialogues, they are limited in conveying empathy and maintaining consistency.
As mental health issues continue to rise globally, there is an increasing demand for accessible and scalable therapeutic solutions. Many individuals currently seek support from Large Language Models (LLMs), even though these models have not been validated for use in counseling services. In this paper, we evaluate LLMs' ability to emulate professional therapists practicing Cognitive Behavioral Therapy (CBT). Using anonymized, transcribed role-play sessions between licensed therapists and clients, we compare two approaches: (1) a generation-only method and (2) a Retrieval-Augmented Generation (RAG) approach using CBT guidelines. We evaluate both proprietary and open-source models for linguistic quality, semantic coherence, and therapeutic fidelity using standard natural language generation (NLG) metrics, natural language inference (NLI), and automated scoring for skills assessment. Our results indicate that while LLMs can generate CBT-like dialogues, they are limited in their ability to convey empathy and maintain consistency.