Leveraging recent advances in Pre-Trained Language Models forEye-Tracking Prediction
This work addresses the challenge of integrating human-derived behavioral data into NLP models for researchers in cognitively inspired AI, but it appears incremental as it combines existing methods like RoBERTa with custom feature engineering.
The paper tackled the problem of predicting eye-tracking gaze features for words in sentences using the ZuCo 1.0 and ZuCo 2.0 datasets, achieving results evaluated with mean absolute error (MAE) and R2 scores, though no concrete numbers are provided in the abstract.
Cognitively inspired Natural Language Pro-cessing uses human-derived behavioral datalike eye-tracking data, which reflect the seman-tic representations of language in the humanbrain to augment the neural nets to solve arange of tasks spanning syntax and semanticswith the aim of teaching machines about lan-guage processing mechanisms. In this paper,we use the ZuCo 1.0 and ZuCo 2.0 dataset con-taining the eye-gaze features to explore differ-ent linguistic models to directly predict thesegaze features for each word with respect to itssentence. We tried different neural networkmodels with the words as inputs to predict thetargets. And after lots of experimentation andfeature engineering finally devised a novel ar-chitecture consisting of RoBERTa Token Clas-sifier with a dense layer on top for languagemodeling and a stand-alone model consistingof dense layers followed by a transformer layerfor the extra features we engineered. Finally,we took the mean of the outputs of both thesemodels to make the final predictions. We eval-uated the models using mean absolute error(MAE) and the R2 score for each target.