Cascaded Transfer: Learning Many Tasks under Budget Constraints
This addresses the challenge of efficient many-task learning for scenarios with limited resources, though it is incremental as it builds on existing transfer learning paradigms.
The paper tackles the problem of learning many related tasks with unknown relationships under budget constraints by introducing Cascaded Transfer Learning, which organizes tasks in a hierarchical tree and allocates training budget along branches, resulting in more accurate and cost-effective adaptation compared to alternatives.
Many-Task Learning refers to the setting where a large number of related tasks need to be learned, the exact relationships between tasks are not known. We introduce the Cascaded Transfer Learning, a novel many-task transfer learning paradigm where information (e.g. model parameters) cascades hierarchically through tasks that are learned by individual models of the same class, while respecting given budget constraints. The cascade is organized as a rooted tree that specifies the order in which tasks are learned and refined. We design a cascaded transfer mechanism deployed over a minimum spanning tree structure that connects the tasks according to a suitable distance measure, and allocates the available training budget along its branches. Experiments on synthetic and real many-task settings show that the resulting method enables more accurate and cost effective adaptation across large task collections compared to alternative approaches.