Eye on the Target: Eye Tracking Meets Rodent Tracking
This work addresses the need for efficient and accurate automation in animal behavior analysis for scientific research, representing an incremental improvement in segmentation methods.
The paper tackled the problem of labor-intensive and subjective manual annotation of animal behavior in video by proposing a pipeline that uses eye-tracking data to generate prompts for zero-shot segmentation, achieving a 70.6% improvement in Jaccard Index from 38.8 to 66.2 on a rats dataset.
Analyzing animal behavior from video recordings is crucial for scientific research, yet manual annotation remains labor-intensive and prone to subjectivity. Efficient segmentation methods are needed to automate this process while maintaining high accuracy. In this work, we propose a novel pipeline that utilizes eye-tracking data from Aria glasses to generate prompt points, which are then used to produce segmentation masks via a fast zero-shot segmentation model. Additionally, we apply post-processing to refine the prompts, leading to improved segmentation quality. Through our approach, we demonstrate that combining eye-tracking-based annotation with smart prompt refinement can enhance segmentation accuracy, achieving an improvement of 70.6% from 38.8 to 66.2 in the Jaccard Index for segmentation results in the rats dataset.