CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection
This addresses the critical need for reliable crisis detection in AI interactions to prevent serious consequences, though it is incremental as it builds on prior work by expanding crisis types and adding temporal labels.
The authors tackled the challenge of detecting diverse mental health crisis situations in language model interactions by introducing CRADLE BENCH, a clinician-annotated benchmark covering seven crisis types with temporal labels, resulting in a dataset of 600 evaluation and 420 development examples, plus a training corpus of around 4K examples that outperformed single-model annotation.
Detecting mental health crisis situations such as suicide ideation, rape, domestic violence, child abuse, and sexual harassment is a critical yet underexplored challenge for language models. When such situations arise during user--model interactions, models must reliably flag them, as failure to do so can have serious consequences. In this work, we introduce CRADLE BENCH, a benchmark for multi-faceted crisis detection. Unlike previous efforts that focus on a limited set of crisis types, our benchmark covers seven types defined in line with clinical standards and is the first to incorporate temporal labels. Our benchmark provides 600 clinician-annotated evaluation examples and 420 development examples, together with a training corpus of around 4K examples automatically labeled using a majority-vote ensemble of multiple language models, which significantly outperforms single-model annotation. We further fine-tune six crisis detection models on subsets defined by consensus and unanimous ensemble agreement, providing complementary models trained under different agreement criteria.