A Mechanism-Driven Theory of Phase Transitions in Active Learning
For active learning practitioners, this provides a unified framework to understand and predict budget-dependent performance, enabling transition-aware algorithm design.
This paper develops a mechanism-driven theory that explains active learning budget regimes as shifts in the dominant generalization mechanism, proving that such shifts are structurally unavoidable. It introduces a tripartite taxonomy (data-driven, transition, model-driven phases) and shows that AL efficiency depends on aligning strategy inductive bias with the active bottleneck, with self-supervised representations shifting transitions earlier.
Active learning (AL) performance is known to be budget-dependent, yet regimes are typically defined by heuristic label counts that fail to generalize across datasets or architectures. We characterize AL dynamics by reframing budget regimes as shifts in the dominant generalization mechanism. By reinterpreting PAC-style risk components as dynamic interacting terms, we prove that dominance shifts are structurally unavoidable, creating a moving bottleneck for generalization. We operationalize this using measurable proxies and a segmented regression procedure to identify a tripartite taxonomy: data-driven, transition, and model-driven phases. Our framework explains the long-standing observation that representativeness, coverage, and uncertainty strategies excel at different stages. Experiments across natural and medical imaging show that AL efficiency depends on the alignment between the strategy's inductive bias and the active bottleneck. Moreover, self-supervised representation shift transitions earlier along the labeling trajectory, highlighting the role of representation quality in shaping AL dynamics. Overall, this work provides a unified framework for the next generation of transition-aware AL algorithms.