Block Neural Network Avoids Catastrophic Forgetting When Learning Multiple Task
This work addresses the issue of forgetting in neural networks for multi-task learning, but it appears incremental as it builds on existing sequential learning paradigms.
The authors tackled the problem of catastrophic forgetting in sequential learning by proposing a deep feed-forward network architecture that reuses features from previous tasks, requiring fewer computational resources and less data for new tasks.
In the present work we propose a Deep Feed Forward network architecture which can be trained according to a sequential learning paradigm, where tasks of increasing difficulty are learned sequentially, yet avoiding catastrophic forgetting. The proposed architecture can re-use the features learned on previous tasks in a new task when the old tasks and the new one are related. The architecture needs fewer computational resources (neurons and connections) and less data for learning the new task than a network trained from scratch