From Conversation to Automation: Leveraging LLMs for Problem-Solving Therapy Analysis
This work addresses the need for scalable tools to automate therapy dialogue analysis for mental health interventions, though it appears incremental as it applies existing LLMs to a new domain.
The researchers tackled the problem of analyzing problem-solving therapy (PST) sessions by developing a framework to annotate therapy transcripts and using LLMs to identify strategies, with GPT-4o achieving the highest accuracy of 0.76.
Problem-solving therapy (PST) is a structured psychological approach that helps individuals manage stress and resolve personal issues by guiding them through problem identification, solution brainstorming, decision-making, and outcome evaluation. As mental health care increasingly adopts technologies like chatbots and large language models (LLMs), it is important to thoroughly understand how each session of PST is conducted before attempting to automate it. We developed a comprehensive framework for PST annotation using established PST Core Strategies and a set of novel Facilitative Strategies to analyze a corpus of real-world therapy transcripts to determine which strategies are most prevalent. Using various LLMs and transformer-based models, we found that GPT-4o outperformed all models, achieving the highest accuracy (0.76) in identifying all strategies. To gain deeper insights, we examined how strategies are applied by analyzing Therapeutic Dynamics (autonomy, self-disclosure, and metaphor), and linguistic patterns within our labeled data. Our research highlights LLMs' potential to automate therapy dialogue analysis, offering a scalable tool for mental health interventions. Our framework enhances PST by improving accessibility, effectiveness, and personalized support for therapists.