Continual Learning by Three-Phase Consolidation
This addresses the challenge of continual learning for AI systems, but it appears incremental as it builds on existing methods to improve performance.
The paper tackles the problem of continual learning by introducing TPC (Three-Phase Consolidation), a method that learns new classes while controlling forgetting of previous knowledge, and demonstrates accuracy and efficiency advantages over existing approaches in experiments on complex datasets.
TPC (Three-Phase Consolidation) is here introduced as a simple but effective approach to continually learn new classes (and/or instances of known classes) while controlling forgetting of previous knowledge. Each experience (a.k.a. task) is learned in three phases characterized by different rules and learning dynamics, aimed at removing the class-bias problem (due to class unbalancing) and limiting gradient-based corrections to prevent forgetting of underrepresented classes. Several experiments on complex datasets demonstrate its accuracy and efficiency advantages over competitive existing approaches. The algorithm and all the results presented in this paper are fully reproducible thanks to its publication on the Avalanche open framework for continual learning.