Integrating Supervised Extractive and Generative Language Models for Suicide Risk Evidence Summarization
This work addresses suicide risk assessment for mental health applications, but it is incremental as it combines existing models for a specific task.
The authors tackled the problem of summarizing evidence for suicide risk by integrating supervised extractive and generative language models, achieving 1st place in highlight extraction and 10th in summary generation in the CLPsych 2024 shared task.
We propose a method that integrates supervised extractive and generative language models for providing supporting evidence of suicide risk in the CLPsych 2024 shared task. Our approach comprises three steps. Initially, we construct a BERT-based model for estimating sentence-level suicide risk and negative sentiment. Next, we precisely identify high suicide risk sentences by emphasizing elevated probabilities of both suicide risk and negative sentiment. Finally, we integrate generative summaries using the MentaLLaMa framework and extractive summaries from identified high suicide risk sentences and a specialized dictionary of suicidal risk words. SophiaADS, our team, achieved 1st place for highlight extraction and ranked 10th for summary generation, both based on recall and consistency metrics, respectively.