On regret bounds for continual single-index learning
This work addresses continual learning challenges for researchers in online and transfer learning, but it appears incremental as it extends existing single-index models to a new setting without broad SOTA claims.
The paper tackles the problem of generalizing single-index models to continual learning, where tasks arrive sequentially with online data, by proposing a randomized strategy that learns a common single-index for all tasks and task-specific link functions, resulting in proven regret bounds under various loss function assumptions.
In this paper, we generalize the problem of single-index model to the context of continual learning in which a learner is challenged with a sequence of tasks one by one and the dataset of each task is revealed in an online fashion. We propose a randomized strategy that is able to learn a common single-index (meta-parameter) for all tasks and a specific link function for each task. The common single-index allows to transfer the information gained from the previous tasks to a new one. We provide a rigorous theoretical analysis of our proposed strategy by proving some regret bounds under different assumption on the loss function.