Assessing the Alignment of Popular CNNs to the Brain for Valence Appraisal
This addresses the problem for psychologists and AI researchers interested in brain-inspired models, but it is incremental as it extends prior work on CNN-brain correspondences to a more complex domain.
The paper tackled the problem of assessing whether popular CNNs align with human brain processes for social cognition, specifically valence appraisal, and found that CNNs struggle to reflect higher-order brain processing beyond simple visual tasks, with correlation analyses showing limited alignment.
Convolutional Neural Networks (CNNs) are a popular type of computer model that have proven their worth in many computer vision tasks. Moreover, they form an interesting study object for the field of psychology, with shown correspondences between the workings of CNNs and the human brain. However, these correspondences have so far mostly been studied in the context of general visual perception. In contrast, this paper explores to what extent this correspondence also holds for a more complex brain process, namely social cognition. To this end, we assess the alignment between popular CNN architectures and both human behavioral and fMRI data for image valence appraisal through a correlation analysis. We show that for this task CNNs struggle to go beyond simple visual processing, and do not seem to reflect higher-order brain processing. Furthermore, we present Object2Brain, a novel framework that combines GradCAM and object detection at the CNN-filter level with the aforementioned correlation analysis to study the influence of different object classes on the CNN-to-human correlations. Despite similar correlation trends, different CNN architectures are shown to display different object class sensitivities.