Adaptive Variational Continual Learning via Task-Heuristic Modelling
This is an incremental improvement for continual learning practitioners who need more adaptive models.
The paper tackles the problem of hyperparameter tuning in variational continual learning by proposing AutoVCL, which automatically adjusts hyperparameters based on task difficulty and similarity. The result shows that AutoVCL outperforms the standard GVCL model with fixed hyperparameters.
Variational continual learning (VCL) is a turn-key learning algorithm that has state-of-the-art performance among the best continual learning models. In our work, we explore an extension of the generalized variational continual learning (GVCL) model, named AutoVCL, which combines task heuristics for informed learning and model optimization. We demonstrate that our model outperforms the standard GVCL with fixed hyperparameters, benefiting from the automatic adjustment of the hyperparameter based on the difficulty and similarity of the incoming task compared to the previous tasks.