Human face perception reflects inverse-generative and naturalistic discriminative objectives
For cognitive scientists and AI researchers, this work identifies computational objectives that align with human face perception, narrowing down plausible mechanisms.
The study compared six neural network models trained on different tasks to human face-dissimilarity judgments, finding that models prioritizing invariant structures (inverse rendering, face identification, object classification) best matched human perception, with natural-image-trained models outperforming synthetic-trained ones.
The perceptual representations supporting our ability to recognize faces remain a computational mystery. Deep neural networks offer mechanistic hypotheses for human face perception, but theoretically distinct models often make indistinguishable representational predictions for randomly sampled faces. To expose diagnostic differences among these hypotheses, we compared six neural network models sharing an architecture but trained on distinct tasks, using face pairs optimized to elicit contrasting model predictions ("controversial" pairs) alongside randomly sampled pairs. We tested model predictions against face-dissimilarity judgments from 864 human participants across stimulus sets differing in realism and pose variation. Models prioritizing high-level, invariant structures (trained via inverse rendering, face identification, or object classification) most robustly matched human judgments. Furthermore, models trained on natural images typically outperformed synthetic-trained counterparts. Together, these findings suggest that human face perception is shaped by mechanisms that infer latent causes of facial appearance, discount nuisance variation, and are tuned by natural image statistics.