Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss
This addresses plasticity loss for neural networks in continual learning and reinforcement learning, representing a novel method for a known bottleneck.
The paper tackles plasticity loss in neural networks by introducing Activation by Interval-wise Dropout (AID), a method that applies Dropout with varying probabilities across preactivation intervals to maintain adaptability, showing effectiveness in continual learning on datasets like CIFAR10 and CIFAR100 and improving reinforcement learning in the Arcade Learning Environment.
Plasticity loss, a critical challenge in neural network training, limits a model's ability to adapt to new tasks or shifts in data distribution. This paper introduces AID (Activation by Interval-wise Dropout), a novel method inspired by Dropout, designed to address plasticity loss. Unlike Dropout, AID generates subnetworks by applying Dropout with different probabilities on each preactivation interval. Theoretical analysis reveals that AID regularizes the network, promoting behavior analogous to that of deep linear networks, which do not suffer from plasticity loss. We validate the effectiveness of AID in maintaining plasticity across various benchmarks, including continual learning tasks on standard image classification datasets such as CIFAR10, CIFAR100, and TinyImageNet. Furthermore, we show that AID enhances reinforcement learning performance in the Arcade Learning Environment benchmark.