Evaluating Subjective Cognitive Appraisals of Emotions from Large Language Models
This addresses the gap in automatic prediction of cognitive appraisals for researchers in psychology and NLP, but it is incremental as it builds on existing emotion detection work.
This work tackled the problem of automatically predicting cognitive appraisals of emotions by creating CovidET-Appraisals, a comprehensive dataset with 24 appraisal dimensions and rationales across 241 Reddit posts, and found that while the best models perform well, open-sourced LLMs fall short, presenting a new challenge for emotionally intelligent models.
The emotions we experience involve complex processes; besides physiological aspects, research in psychology has studied cognitive appraisals where people assess their situations subjectively, according to their own values (Scherer, 2005). Thus, the same situation can often result in different emotional experiences. While the detection of emotion is a well-established task, there is very limited work so far on the automatic prediction of cognitive appraisals. This work fills the gap by presenting CovidET-Appraisals, the most comprehensive dataset to-date that assesses 24 appraisal dimensions, each with a natural language rationale, across 241 Reddit posts. CovidET-Appraisals presents an ideal testbed to evaluate the ability of large language models -- excelling at a wide range of NLP tasks -- to automatically assess and explain cognitive appraisals. We found that while the best models are performant, open-sourced LLMs fall short at this task, presenting a new challenge in the future development of emotionally intelligent models. We release our dataset at https://github.com/honglizhan/CovidET-Appraisals-Public.