Edu-EmotionNet: Cross-Modality Attention Alignment with Temporal Feedback Loops
This addresses the problem of improving engagement and personalized instruction in online education by providing a more reliable emotion recognition system, though it appears incremental as it builds on prior multimodal and temporal modeling work.
The paper tackled robust emotion recognition in online education by introducing Edu-EmotionNet, which models temporal emotion evolution and modality reliability, achieving state-of-the-art performance on educational subsets of IEMOCAP and MOSEI with strong robustness to missing or noisy modalities.
Understanding learner emotions in online education is critical for improving engagement and personalized instruction. While prior work in emotion recognition has explored multimodal fusion and temporal modeling, existing methods often rely on static fusion strategies and assume that modality inputs are consistently reliable, which is rarely the case in real-world learning environments. We introduce Edu-EmotionNet, a novel framework that jointly models temporal emotion evolution and modality reliability for robust affect recognition. Our model incorporates three key components: a Cross-Modality Attention Alignment (CMAA) module for dynamic cross-modal context sharing, a Modality Importance Estimator (MIE) that assigns confidence-based weights to each modality at every time step, and a Temporal Feedback Loop (TFL) that leverages previous predictions to enforce temporal consistency. Evaluated on educational subsets of IEMOCAP and MOSEI, re-annotated for confusion, curiosity, boredom, and frustration, Edu-EmotionNet achieves state-of-the-art performance and demonstrates strong robustness to missing or noisy modalities. Visualizations confirm its ability to capture emotional transitions and adaptively prioritize reliable signals, making it well suited for deployment in real-time learning systems