Diagnosing Catastrophe: Large parts of accuracy loss in continual learning can be accounted for by readout misalignment
This addresses the problem of catastrophic forgetting for deep learning applications requiring continuous updates, potentially aiding in aligning models to robust biological vision, but it is incremental as it builds on existing understanding of the phenomenon.
The paper investigated the representational changes underlying catastrophic forgetting in artificial neural networks and found that the largest component of accuracy loss is due to misalignment between hidden representations and readout layers, with representational geometry partially conserved and only a small part of information irrecoverably lost.
Unlike primates, training artificial neural networks on changing data distributions leads to a rapid decrease in performance on old tasks. This phenomenon is commonly referred to as catastrophic forgetting. In this paper, we investigate the representational changes that underlie this performance decrease and identify three distinct processes that together account for the phenomenon. The largest component is a misalignment between hidden representations and readout layers. Misalignment occurs due to learning on additional tasks and causes internal representations to shift. Representational geometry is partially conserved under this misalignment and only a small part of the information is irrecoverably lost. All types of representational changes scale with the dimensionality of hidden representations. These insights have implications for deep learning applications that need to be continuously updated, but may also aid aligning ANN models to the rather robust biological vision.