What Can AutoML Do For Continual Learning?
It identifies opportunities for AutoML to enhance continual learning, potentially benefiting AI systems that need to adapt over time, but it is incremental as it outlines directions rather than presenting new methods.
This position paper explores how AutoML can address limitations in continual learning, such as frozen architectures and hyperparameters, by proposing research directions to make incremental learners more dynamic and adaptable to diverse real-world tasks.
This position paper outlines the potential of AutoML for incremental (continual) learning to encourage more research in this direction. Incremental learning involves incorporating new data from a stream of tasks and distributions to learn enhanced deep representations and adapt better to new tasks. However, a significant limitation of incremental learners is that most current techniques freeze the backbone architecture, hyperparameters, and the order & structure of the learning tasks throughout the learning and adaptation process. We strongly believe that AutoML offers promising solutions to address these limitations, enabling incremental learning to adapt to more diverse real-world tasks. Therefore, instead of directly proposing a new method, this paper takes a step back by posing the question: "What can AutoML do for incremental learning?" We outline three key areas of research that can contribute to making incremental learners more dynamic, highlighting concrete opportunities to apply AutoML methods in novel ways as well as entirely new challenges for AutoML research.