FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff
This addresses the challenge of tuning reinitialization methods for continual learning, though it appears incremental as it builds on standard reinitialization approaches.
The paper tackles the stability-plasticity tradeoff in deep neural networks trained on nonstationary data by proposing FIRE, a reinitialization method that balances retaining prior knowledge and adapting to new tasks, achieving consistent performance gains across continual visual learning, language modeling, and reinforcement learning domains.
Deep neural networks trained on nonstationary data must balance stability (i.e., retaining prior knowledge) and plasticity (i.e., adapting to new tasks). Standard reinitialization methods, which reinitialize weights toward their original values, are widely used but difficult to tune: conservative reinitializations fail to restore plasticity, while aggressive ones erase useful knowledge. We propose FIRE, a principled reinitialization method that explicitly balances the stability-plasticity tradeoff. FIRE quantifies stability through Squared Frobenius Error (SFE), measuring proximity to past weights, and plasticity through Deviation from Isometry (DfI), reflecting weight isotropy. The reinitialization point is obtained by solving a constrained optimization problem, minimizing SFE subject to DfI being zero, which is efficiently approximated by Newton-Schulz iteration. FIRE is evaluated on continual visual learning (CIFAR-10 with ResNet-18), language modeling (OpenWebText with GPT-0.1B), and reinforcement learning (HumanoidBench with SAC and Atari games with DQN). Across all domains, FIRE consistently outperforms both naive training without intervention and standard reinitialization methods, demonstrating effective balancing of the stability-plasticity tradeoff.