Generating Accurate Virtual Examples For Lifelong Machine Learning
This work addresses the challenge of catastrophic forgetting in lifelong learning systems, offering a method for task rehearsal that is incremental in nature.
The paper tackles the lifelong machine learning retention problem by generating accurate virtual examples from a trained Restricted Boltzmann Machine using reconstruction error, achieving successful knowledge consolidation without retaining prior task data.
Lifelong machine learning (LML) is an area of machine learning research concerned with human-like persistent and cumulative nature of learning. LML system's objective is consolidating new information into an existing machine learning model without catastrophically disrupting the prior information. Our research addresses this LML retention problem for creating a knowledge consolidation network through task rehearsal without retaining the prior task's training examples. We discovered that the training data reconstruction error from a trained Restricted Boltzmann Machine can be successfully used to generate accurate virtual examples from the reconstructed set of a uniform random set of examples given to the trained model. We also defined a measure for comparing the probability distributions of two datasets given to a trained network model based on their reconstruction mean square errors.