Patch-level Gaze Distribution Prediction for Gaze Following
This work addresses gaze following for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles the problem of gaze following by addressing limitations in existing methods that use pixel-level heatmap regression with strict constraints and separate in/out prediction. The proposed patch distribution prediction method improves performance on both gaze target localization and in/out prediction subtasks on public datasets.
Gaze following aims to predict where a person is looking in a scene, by predicting the target location, or indicating that the target is located outside the image. Recent works detect the gaze target by training a heatmap regression task with a pixel-wise mean-square error (MSE) loss, while formulating the in/out prediction task as a binary classification task. This training formulation puts a strict, pixel-level constraint in higher resolution on the single annotation available in training, and does not consider annotation variance and the correlation between the two subtasks. To address these issues, we introduce the patch distribution prediction (PDP) method. We replace the in/out prediction branch in previous models with the PDP branch, by predicting a patch-level gaze distribution that also considers the outside cases. Experiments show that our model regularizes the MSE loss by predicting better heatmap distributions on images with larger annotation variances, meanwhile bridging the gap between the target prediction and in/out prediction subtasks, showing a significant improvement in performance on both subtasks on public gaze following datasets.