If Eleanor Rigby Had Met ChatGPT: A Study on Loneliness in a Post-LLM World
This highlights ethical and legal risks for vulnerable individuals using general-purpose LLMs for loneliness, potentially leading to harm like radicalization or isolation.
The study examined how users interact with ChatGPT for loneliness-related support, finding that 37% sought advice or validation and engagement was good, but it failed in sensitive scenarios like suicidal ideation and had a 35% higher incidence of toxic content, with women 22 times more likely to be targeted.
Warning: this paper discusses content related, but not limited to, violence, sex, and suicide. Loneliness, or the lack of fulfilling relationships, significantly impacts a person's mental and physical well-being and is prevalent worldwide. Previous research suggests that large language models (LLMs) may help mitigate loneliness. However, we argue that the use of widespread LLMs in services like ChatGPT is more prevalent--and riskier, as they are not designed for this purpose. To explore this, we analysed user interactions with ChatGPT outside of its marketed use as a task-oriented assistant. In dialogues classified as lonely, users frequently (37%) sought advice or validation, and received good engagement. However, ChatGPT failed in sensitive scenarios, like responding appropriately to suicidal ideation or trauma. We also observed a 35% higher incidence of toxic content, with women being 22x more likely to be targeted than men. Our findings underscore ethical and legal questions about this technology, and note risks like radicalisation or further isolation. We conclude with recommendations to research and industry to address loneliness.