Person Perception Biases Exposed: Revisiting the First Impressions Dataset
This work highlights how inherent human biases in perception can significantly influence data labeling for subjective tasks, posing a problem for the computer vision and machine learning communities that rely on such datasets for practical applications.
This study re-examines the ChaLearn First Impressions dataset, revealing that person perception biases related to perceived gender, ethnicity, age, and face attractiveness are present in the original pairwise annotations. It also demonstrates that the conversion mechanism from pairwise annotations to continuous values can amplify these biases.
This work revisits the ChaLearn First Impressions database, annotated for personality perception using pairwise comparisons via crowdsourcing. We analyse for the first time the original pairwise annotations, and reveal existing person perception biases associated to perceived attributes like gender, ethnicity, age and face attractiveness. We show how person perception bias can influence data labelling of a subjective task, which has received little attention from the computer vision and machine learning communities by now. We further show that the mechanism used to convert pairwise annotations to continuous values may magnify the biases if no special treatment is considered. The findings of this study are relevant for the computer vision community that is still creating new datasets on subjective tasks, and using them for practical applications, ignoring these perceptual biases.