A Novel Approach to Lifelong Learning: The Plastic Support Structure
This addresses the challenge of enabling AI systems to learn continuously over time, though it appears incremental as it builds on existing lifelong learning architectures.
The paper tackles the problem of catastrophic forgetting in lifelong learning by introducing a Plastic Support Structure (PSS) that allows networks to expand capacity for new tasks without losing old ones, achieving performance similar to existing methods but with fewer parameters and no prior task knowledge.
We propose a novel approach to lifelong learning, introducing a compact encapsulated support structure which endows a network with the capability to expand its capacity as needed to learn new tasks while preventing the loss of learned tasks. This is achieved by splitting neurons with high semantic drift and constructing an adjacent network to encode the new tasks at hand. We call this the Plastic Support Structure (PSS), it is a compact structure to learn new tasks that cannot be efficiently encoded in the existing structure of the network. We validate the PSS on public datasets against existing lifelong learning architectures, showing it performs similarly to them but without prior knowledge of the task and in some cases with fewer parameters and in a more understandable fashion where the PSS is an encapsulated container for specific features related to specific tasks, thus making it an ideal "add-on" solution for endowing a network to learn more tasks.