CSGaze: Context-aware Social Gaze Prediction
This work addresses social gaze prediction for conversational analysis, which is incremental as it builds on existing methods by incorporating contextual cues.
The paper tackled the problem of predicting social gaze patterns in conversational interactions by developing CSGaze, a context-aware multimodal approach that uses facial and scene information, achieving competitive performance with state-of-the-art methods on datasets like GP-Static, UCO-LAEO, and AVA-LAEO.
A person's gaze offers valuable insights into their focus of attention, level of social engagement, and confidence. In this work, we investigate how contextual cues combined with visual scene and facial information can be effectively utilized to predict and interpret social gaze patterns during conversational interactions. We introduce CSGaze, a context aware multimodal approach that leverages facial, scene information as complementary inputs to enhance social gaze pattern prediction from multi-person images. The model also incorporates a fine-grained attention mechanism centered on the principal speaker, which helps in better modeling social gaze dynamics. Experimental results show that CSGaze performs competitively with state-of-the-art methods on GP-Static, UCO-LAEO and AVA-LAEO. Our findings highlight the role of contextual cues in improving social gaze prediction. Additionally, we provide initial explainability through generated attention scores, offering insights into the model's decision-making process. We also demonstrate our model's generalizability by testing our model on open set datasets that demonstrating its robustness across diverse scenarios.