REG: A Regularization Optimizer for Robust Training Dynamics
This work addresses training dynamics issues for LLM developers, offering an incremental improvement over existing optimizers.
The authors tackled the problem of training instability and incompatibility in structure-aware optimizers like Muon for Large Language Models by proposing REG, which replaces Muon's matrix sign function with a Row-and-Column-Scaling operator, achieving superior performance and stability over AdamW while maintaining consistency during fine-tuning.
Optimizers are crucial for the efficient training of Large Language Models (LLMs). While AdamW is the de facto standard, recent structure-aware optimizers like Muon have emerged, which regularize gradient updates by operating on entire weight matrices. The Muon optimizer balances the gradient updates along all the directions. However, Muon's reliance on the matrix sign function can lead to training instability, exhibits incompatibility when fine-tuning models pre-trained with AdamW. To address these limitations, we propose \textbf{REG}, a novel optimizer that replaces Muon's aggressive matrix sign operator with the Row-and-Column-Scaling (RACS) operator. Theoretically grounded in balancing a matrix, the RACS operator regularizes the update steps in a less drastic manner, making it simpler to implement and more compatible with established training dynamics. Through extensive empirical experiments on LLM training, we demonstrate that our REG optimizer not only achieves superior performance and stability over AdamW, but also maintains consistency with the AdamW training paradigm. This consistency is particularly evident during the fine-tuning stage, where REG optimizer avoids the performance degradation observed with Muon.