Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization
This provides theoretical insights into optimization for neural networks, but it is incremental as it builds on existing overparameterization studies.
The paper tackled the geometry of loss landscapes near global minima for two-layer neural networks, showing that global minima become separated and gradient flow converges locally in overparameterized regimes.
Under mild assumptions, we investigate the geometry of the loss landscape for two-layer neural networks in the vicinity of global minima. Utilizing novel techniques, we demonstrate: (i) how global minima with zero generalization error become geometrically separated from other global minima as the sample size grows; and (ii) the local convergence properties and rate of gradient flow dynamics. Our results indicate that two-layer neural networks can be locally recovered in the regime of overparameterization.