To Trust, or Not to Trust? A Study of Human Bias in Automated Video Interview Assessments
This addresses potential bias in training data for automated assessment systems, which could impact fairness in hiring or evaluation processes, but it is incremental as it focuses on preliminary evidence.
The study investigated whether human annotations for training supervised systems in automated video interview assessments are biased, finding preliminary evidence of visual biases in ratings.
Supervised systems require human labels for training. But, are humans themselves always impartial during the annotation process? We examine this question in the context of automated assessment of human behavioral tasks. Specifically, we investigate whether human ratings themselves can be trusted at their face value when scoring video-based structured interviews, and whether such ratings can impact machine learning models that use them as training data. We present preliminary empirical evidence that indicates there might be biases in such annotations, most of which are visual in nature.