Behavioral Experiments for Understanding Catastrophic Forgetting
This work addresses the problem of catastrophic forgetting for researchers in machine learning, offering an incremental approach by adapting existing psychological methods to neural networks.
The paper tackled catastrophic forgetting in neural networks by applying behavioral experiments from psychology, revealing new insights into its behavior through controlled experiments with two-layer ReLU networks.
In this paper we explore whether the fundamental tool of experimental psychology, the behavioral experiment, has the power to generate insight not only into humans and animals, but artificial systems too. We apply the techniques of experimental psychology to investigating catastrophic forgetting in neural networks. We present a series of controlled experiments with two-layer ReLU networks, and exploratory results revealing a new understanding of the behavior of catastrophic forgetting. Alongside our empirical findings, we demonstrate an alternative, behavior-first approach to investigating neural network phenomena.