Unveiling the Basin-Like Loss Landscape in Large Language Models
This work addresses the problem of understanding and improving the stability and robustness of large language models for AI researchers and practitioners, though it is incremental in building on existing loss landscape concepts.
The study discovered that large language models develop basin-like loss landscapes as they scale, where models are resilient to parameter perturbations within expansive stability regions but collapse outside them, with pre-training forming basic capability basins and fine-tuning creating specific ones, and theoretical analysis showed basin size bounds performance degradation and enhances robustness.
We discover the emergence of \textit{basins} in the loss landscape of large language models. As model scale increases, LLMs become progressively more resilient to random perturbations in the parameter space, giving rise to expansive stability regions where models exhibit nearly identical performance, but outside of which their capabilities collapse. We observe that pre-training creates a \textit{basic capability} basin, and subsequent alignment fine-tuning forms \textit{specific capability} basins (e.g., safety, math, coding). Thus, we argue that benign fine-tuning confined to the basin should preserve prior capabilities. Besides, we also analyze the loss landscape for worst-case directions, which is consistently sharp and detrimental. We find that adversarial fine-tuning moves along the nearly worst-case directions, thus rapidly degrading model capabilities. Finally, we provide a theoretical analysis demonstrating that the basin size bounds the performance degradation of any fine-tuning, including the adversarial ones, while also guaranteeing the model robustness w.r.t. input perturbations, suggesting the benefit of enlarging basins.