Adapting, Fast and Slow: Transportable Circuits for Few-Shot Learning
This work addresses domain adaptation for few-shot learning, offering a novel approach based on causal graphs, but it appears incremental as it builds on existing transportability theory.
The paper tackles the problem of few-shot learning across domains by proposing an algorithm that uses causal transportability theory to generalize from source to target domains, achieving theoretical characterization of learnable tasks and corroborating results with simulations.
Generalization across the domains is not possible without asserting a structure that constrains the unseen target domain w.r.t. the source domain. Building on causal transportability theory, we design an algorithm for zero-shot compositional generalization which relies on access to qualitative domain knowledge in form of a causal graph for intra-domain structure and discrepancies oracle for inter-domain mechanism sharing. \textit{Circuit-TR} learns a collection of modules (i.e., local predictors) from the source data, and transport/compose them to obtain a circuit for prediction in the target domain if the causal structure licenses. Furthermore, circuit transportability enables us to design a supervised domain adaptation scheme that operates without access to an explicit causal structure, and instead uses limited target data. Our theoretical results characterize classes of few-shot learnable tasks in terms of graphical circuit transportability criteria, and connects few-shot generalizability with the established notion of circuit size complexity; controlled simulations corroborate our theoretical results.