Natural Way to Overcome the Catastrophic Forgetting in Neural Networks
This addresses the problem of preserving learned skills during subsequent training for neural network users, but appears incremental as it builds on existing methods.
The paper tackles catastrophic forgetting in neural networks by proposing an alternative method based on total absolute signal passing through connections, which is simple to implement and biologically inspired.
Not so long ago, a method was discovered that successfully overcomes the catastrophic forgetting in neural networks. Although we know about the cases of using this method to preserve skills when adapting pre-trained networks to particular tasks, it has not obtained widespread distribution yet. In this paper, we would like to propose an alternative method of overcoming catastrophic forgetting based on the total absolute signal passing through each connection in the network. This method has a simple implementation and seems to us essentially close to the processes occurring in the brain of animals to preserve previously learned skills during subsequent learning. We hope that the ease of implementation of this method will serve its wide application.