Effective dimension of machine learning models
This work addresses the challenge of explaining generalization in machine learning models, which is incremental as it builds on existing capacity measures.
The authors tackled the problem of understanding model generalization by proposing the local effective dimension as a capacity measure, which they proved bounds generalization error and correlates well with generalization error on standard datasets.
Making statements about the performance of trained models on tasks involving new data is one of the primary goals of machine learning, i.e., to understand the generalization power of a model. Various capacity measures try to capture this ability, but usually fall short in explaining important characteristics of models that we observe in practice. In this study, we propose the local effective dimension as a capacity measure which seems to correlate well with generalization error on standard data sets. Importantly, we prove that the local effective dimension bounds the generalization error and discuss the aptness of this capacity measure for machine learning models.