How Far Can Sharpness and Complexity Jointly Explain Generalization?
For researchers studying generalization in deep learning, this paper provides a quantitative evaluation of the joint explanatory power of sharpness and complexity, showing that function-oriented definitions expand the explanatory scope but the two-factor view remains incomplete.
This work investigates how well sharpness and complexity jointly explain generalization in deep neural networks, using linear regression and Pareto-based analysis. Function-oriented definitions of these factors improve explanatory power over parameter-level metrics, but the two-factor view is not a complete theory of generalization.
Sharpness and complexity are two central factors in the generalization analysis of deep neural networks. Existing quantitative evaluations of generalization measures have largely focused on individual scalar measures, leaving the joint explanatory power of sharpness and complexity largely unexplored. This work studies how far sharpness and complexity can jointly explain generalization. We use linear regression and introduce a Pareto-based analysis to quantitatively evaluate the joint explanatory power of these two factors. Beyond the existing parameter-level definitions, we further propose realizations of sharpness and complexity that are closer to function space and less dependent on raw parameter representations. We find that function-oriented definitions of these two quantities expand the explanatory scope of the two-factor view beyond what is achieved by existing parameter-level metrics. Overall, our results support the sharpness-complexity perspective as an informative lens for understanding generalization across diverse settings. At the same time, the remaining failures indicate that whether this two-factor view can serve as a complete theory of generalization remains open.