Trust Modeling in Counseling Conversations: A Benchmark Study
This work addresses the need for better assessment of therapeutic bonds in counseling, which is crucial for improving patient care, though it is incremental as it builds on existing dialogue modeling with a new focus on trust.
The study tackled the problem of modeling trust in mental health counseling conversations by introducing a novel dataset, MENTAL-TRUST, with expert-annotated trust levels, and established a benchmark, TrustBench, for ordinal classification, achieving evaluation across multiple metrics to analyze trust evolution.
In mental health counseling, a variety of earlier studies have focused on dialogue modeling. However, most of these studies give limited to no emphasis on the quality of interaction between a patient and a therapist. The therapeutic bond between a patient and a therapist directly correlates with effective mental health counseling. It involves developing the patient's trust on the therapist over the course of counseling. To assess the therapeutic bond in counseling, we introduce trust as a therapist-assistive metric. Our definition of trust involves patients' willingness and openness to express themselves and, consequently, receive better care. We conceptualize it as a dynamic trajectory observable through textual interactions during the counseling. To facilitate trust modeling, we present MENTAL-TRUST, a novel counseling dataset comprising manual annotation of 212 counseling sessions with first-of-its-kind seven expert-verified ordinal trust levels. We project our problem statement as an ordinal classification task for trust quantification and propose a new benchmark, TrustBench, comprising a suite of classical and state-of-the-art language models on MENTAL-TRUST. We evaluate the performance across a suite of metrics and lay out an exhaustive set of findings. Our study aims to unfold how trust evolves in therapeutic interactions.