Deconfounded Representation Similarity for Comparison of Neural Networks
This addresses a methodological issue for researchers comparing neural networks, offering an incremental improvement to existing metrics.
The paper tackled the problem of similarity metrics for neural network representations being confounded by input data structure, which leads to misleading comparisons. They introduced a deconfounding method using covariate adjustment regression, showing it improves detection of semantic similarity, consistency with domain similarities in transfer learning, and correlation with out-of-distribution accuracy.
Similarity metrics such as representational similarity analysis (RSA) and centered kernel alignment (CKA) have been used to compare layer-wise representations between neural networks. However, these metrics are confounded by the population structure of data items in the input space, leading to spuriously high similarity for even completely random neural networks and inconsistent domain relations in transfer learning. We introduce a simple and generally applicable fix to adjust for the confounder with covariate adjustment regression, which retains the intuitive invariance properties of the original similarity measures. We show that deconfounding the similarity metrics increases the resolution of detecting semantically similar neural networks. Moreover, in real-world applications, deconfounding improves the consistency of representation similarities with domain similarities in transfer learning, and increases correlation with out-of-distribution accuracy.