Supermasks in Superposition
This addresses the problem of catastrophic forgetting for AI systems requiring lifelong learning, though it is incremental as it builds on prior subnetwork and superposition methods.
The paper tackles catastrophic forgetting in sequential learning by introducing the Supermasks in Superposition (SupSup) model, which uses fixed random networks and task-specific supermasks to achieve good performance on thousands of tasks, with a single gradient step often sufficient to identify the correct mask among 2500 tasks.
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting. Our approach uses a randomly initialized, fixed base network and for each task finds a subnetwork (supermask) that achieves good performance. If task identity is given at test time, the correct subnetwork can be retrieved with minimal memory usage. If not provided, SupSup can infer the task using gradient-based optimization to find a linear superposition of learned supermasks which minimizes the output entropy. In practice we find that a single gradient step is often sufficient to identify the correct mask, even among 2500 tasks. We also showcase two promising extensions. First, SupSup models can be trained entirely without task identity information, as they may detect when they are uncertain about new data and allocate an additional supermask for the new training distribution. Finally the entire, growing set of supermasks can be stored in a constant-sized reservoir by implicitly storing them as attractors in a fixed-sized Hopfield network.