Spectral Thresholds for Identifiability and Stability:Finite-Sample Phase Transitions in High-Dimensional Learning
This work addresses the issue of instability in high-dimensional learning for researchers and practitioners, providing a finite-sample, non-asymptotic criterion that bridges theory with diagnostics, though it is incremental in sharpening classical eigenvalue conditions.
The paper tackles the problem of abrupt model collapse in high-dimensional learning by identifying a critical sample size threshold, proving that stability requires the minimal Fisher eigenvalue to exceed an explicit O(√(d/n)) bound, with synthetic experiments confirming the predicted transition consistent with d/n scaling.
In high-dimensional learning, models remain stable until they collapse abruptly once the sample size falls below a critical level. This instability is not algorithm-specific but a geometric mechanism: when the weakest Fisher eigendirection falls beneath sample-level fluctuations, identifiability fails. Our Fisher Threshold Theorem formalizes this by proving that stability requires the minimal Fisher eigenvalue to exceed an explicit $O(\sqrt{d/n})$ bound. Unlike prior asymptotic or model-specific criteria, this threshold is finite-sample and necessary, marking a sharp phase transition between reliable concentration and inevitable failure. To make the principle constructive, we introduce the Fisher floor, a verifiable spectral regularization robust to smoothing and preconditioning. Synthetic experiments on Gaussian mixtures and logistic models confirm the predicted transition, consistent with $d/n$ scaling. Statistically, the threshold sharpens classical eigenvalue conditions into a non-asymptotic law; learning-theoretically, it defines a spectral sample-complexity frontier, bridging theory with diagnostics for robust high-dimensional inference.