Gi and Pal Scores: Deep Neural Network Generalization Statistics
This work addresses a foundational problem in deep learning for researchers and practitioners by providing new generalization statistics, though it appears incremental as it adapts existing inequality measures.
The authors tackled the lack of theoretical error bounds and consistent measures for deep neural network generalization by introducing Gi-score and Pal-score, which predict generalization gaps with robustness to perturbations.
The field of Deep Learning is rich with empirical evidence of human-like performance on a variety of regression, classification, and control tasks. However, despite these successes, the field lacks strong theoretical error bounds and consistent measures of network generalization and learned invariances. In this work, we introduce two new measures, the Gi-score and Pal-score, that capture a deep neural network's generalization capabilities. Inspired by the Gini coefficient and Palma ratio, measures of income inequality, our statistics are robust measures of a network's invariance to perturbations that accurately predict generalization gaps, i.e., the difference between accuracy on training and test sets.