Improving the Accuracy and Robustness of CNNs Using a Deep CCA Neural Data Regularizer
This work addresses the challenge of enhancing CNN performance for computer vision tasks, offering a novel regularization approach that yields significant improvements, though it is incremental in advancing neural data regularization methods.
The authors tackled the problem of improving CNN accuracy and robustness by developing a neural data regularizer using Deep Canonical Correlation Analysis to align CNN representations with primate visual cortex data, resulting in larger gains in classification accuracy and within-super-class accuracy compared to previous methods, and increased robustness to adversarial attacks.
As convolutional neural networks (CNNs) become more accurate at object recognition, their representations become more similar to the primate visual system. This finding has inspired us and other researchers to ask if the implication also runs the other way: If CNN representations become more brain-like, does the network become more accurate? Previous attempts to address this question showed very modest gains in accuracy, owing in part to limitations of the regularization method. To overcome these limitations, we developed a new neural data regularizer for CNNs that uses Deep Canonical Correlation Analysis (DCCA) to optimize the resemblance of the CNN's image representations to that of the monkey visual cortex. Using this new neural data regularizer, we see much larger performance gains in both classification accuracy and within-super-class accuracy, as compared to the previous state-of-the-art neural data regularizers. These networks are also more robust to adversarial attacks than their unregularized counterparts. Together, these results confirm that neural data regularization can push CNN performance higher, and introduces a new method that obtains a larger performance boost.