LyS at SemEval-2024 Task 3: An Early Prototype for End-to-End Multimodal Emotion Linking as Graph-Based Parsing
This work addresses multimodal emotion cause analysis for conversational AI, but it is incremental as it builds on existing graph-based and transformer methods.
The paper tackled the problem of identifying causal emotion relations in multi-party conversations by developing an end-to-end system using graph-based parsing methods, achieving a ranking of 7th out of 15 submissions in a specific subtask.
This paper describes our participation in SemEval 2024 Task 3, which focused on Multimodal Emotion Cause Analysis in Conversations. We developed an early prototype for an end-to-end system that uses graph-based methods from dependency parsing to identify causal emotion relations in multi-party conversations. Our model comprises a neural transformer-based encoder for contextualizing multimodal conversation data and a graph-based decoder for generating the adjacency matrix scores of the causal graph. We ranked 7th out of 15 valid and official submissions for Subtask 1, using textual inputs only. We also discuss our participation in Subtask 2 during post-evaluation using multi-modal inputs.