PRISM: Structured Optimization via Anisotropic Spectral Shaping
This work addresses the challenge of improving optimization efficiency in machine learning by introducing a novel method for curvature adaptation, though it appears incremental as it builds on existing spectral optimization paradigms.
The paper tackles the problem of enhancing first-order spectral descent methods by integrating partial second-order information, resulting in PRISM, an optimizer that achieves anisotropic spectral shaping with minimal computational overhead and zero additional memory compared to baselines.
We propose PRISM, an optimizer that enhances first-order spectral descent methods like Muon with partial second-order information. It constructs an efficient, low-rank quasi-second-order preconditioner via innovation-augmented polar decomposition. This mechanism enables PRISM to perform anisotropic spectral shaping, which adaptively suppresses updates in high-variance subspaces while preserving update strength in signal-dominated directions. Crucially, this is achieved with minimal computational overhead and zero additional memory compared to first-order baselines. PRISM demonstrates a practical strategy for integrating curvature-adaptive properties into the spectral optimization paradigm.