Large Language Models for Detection of Life-Threatening Texts
This work addresses the need for effective detection of life-threatening language to safeguard individuals and promote mental health, representing an incremental improvement over existing methods.
The paper tackled the problem of detecting life-threatening texts by fine-tuning large language models (LLMs) like Gemma, Mistral, and Llama-2, and found that Mistral and Llama-2 outperformed traditional methods such as bag of words and BERT in both balanced and imbalanced data scenarios.
Detecting life-threatening language is essential for safeguarding individuals in distress, promoting mental health and well-being, and preventing potential harm and loss of life. This paper presents an effective approach to identifying life-threatening texts using large language models (LLMs) and compares them with traditional methods such as bag of words, word embedding, topic modeling, and Bidirectional Encoder Representations from Transformers. We fine-tune three open-source LLMs including Gemma, Mistral, and Llama-2 using their 7B parameter variants on different datasets, which are constructed with class balance, imbalance, and extreme imbalance scenarios. Experimental results demonstrate a strong performance of LLMs against traditional methods. More specifically, Mistral and Llama-2 models are top performers in both balanced and imbalanced data scenarios while Gemma is slightly behind. We employ the upsampling technique to deal with the imbalanced data scenarios and demonstrate that while this method benefits traditional approaches, it does not have as much impact on LLMs. This study demonstrates a great potential of LLMs for real-world life-threatening language detection problems.