Attention Flow: End-to-End Joint Attention Estimation
This work addresses joint attention estimation for applications in human-computer interaction, education, and healthcare, but it is incremental as it builds on existing datasets and methods.
The paper tackles the problem of estimating joint attention in third-person social scene videos by proposing Attention Flow, an end-to-end method that uses saliency-augmented attention maps and novel convolutional attention mechanisms, achieving quantitative and qualitative results on the VideoCoAtt dataset.
This paper addresses the problem of understanding joint attention in third-person social scene videos. Joint attention is the shared gaze behaviour of two or more individuals on an object or an area of interest and has a wide range of applications such as human-computer interaction, educational assessment, treatment of patients with attention disorders, and many more. Our method, Attention Flow, learns joint attention in an end-to-end fashion by using saliency-augmented attention maps and two novel convolutional attention mechanisms that determine to select relevant features and improve joint attention localization. We compare the effect of saliency maps and attention mechanisms and report quantitative and qualitative results on the detection and localization of joint attention in the VideoCoAtt dataset, which contains complex social scenes.