Sequence Transferability and Task Order Selection in Continual Learning
This work addresses the underdeveloped area of task sequence analysis in continual learning, offering incremental improvements for algorithm development.
The paper tackled the problem of task sequence properties affecting model performance in continual learning by proposing two novel measures of sequence transferability and a new method for task order selection, resulting in better performance than random selection.
In continual learning, understanding the properties of task sequences and their relationships to model performance is important for developing advanced algorithms with better accuracy. However, efforts in this direction remain underdeveloped despite encouraging progress in methodology development. In this work, we investigate the impacts of sequence transferability on continual learning and propose two novel measures that capture the total transferability of a task sequence, either in the forward or backward direction. Based on the empirical properties of these measures, we then develop a new method for the task order selection problem in continual learning. Our method can be shown to offer a better performance than the conventional strategy of random task selection.