TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking
This addresses the bottleneck of small datasets for saliency prediction in computer vision, enabling more data-intensive training and reducing overfitting, though it is incremental as it builds on existing crowdsourcing methods.
The paper tackled the problem of expensive and limited gaze data collection for saliency prediction by developing a webcam-based eye tracking system deployed on Amazon Mechanical Turk, resulting in a large-scale saliency dataset comparable to lab data at lower cost.
Traditional eye tracking requires specialized hardware, which means collecting gaze data from many observers is expensive, tedious and slow. Therefore, existing saliency prediction datasets are order-of-magnitudes smaller than typical datasets for other vision recognition tasks. The small size of these datasets limits the potential for training data intensive algorithms, and causes overfitting in benchmark evaluation. To address this deficiency, this paper introduces a webcam-based gaze tracking system that supports large-scale, crowdsourced eye tracking deployed on Amazon Mechanical Turk (AMTurk). By a combination of careful algorithm and gaming protocol design, our system obtains eye tracking data for saliency prediction comparable to data gathered in a traditional lab setting, with relatively lower cost and less effort on the part of the researchers. Using this tool, we build a saliency dataset for a large number of natural images. We will open-source our tool and provide a web server where researchers can upload their images to get eye tracking results from AMTurk.