Can We Understand Plasticity Through Neural Collapse?
This work addresses the problem of understanding and mitigating plasticity loss in neural networks for researchers, though it is incremental as it builds on known phenomena.
The paper investigates the relationship between plasticity loss and neural collapse in deep learning, finding a significant correlation during initial training on the first task, and proposes a regularization method that reduces neural collapse and alleviates plasticity loss in this context.
This paper explores the connection between two recently identified phenomena in deep learning: plasticity loss and neural collapse. We analyze their correlation in different scenarios, revealing a significant association during the initial training phase on the first task. Additionally, we introduce a regularization approach to mitigate neural collapse, demonstrating its effectiveness in alleviating plasticity loss in this specific setting.