How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget
This work addresses a practical challenge in active learning for researchers and practitioners, but it is incremental as it builds on existing strategies with a dynamic selection approach.
The paper tackles the problem of selecting the best active learning strategy for specific conditions and budgets, proposing a derivative-based method that dynamically identifies optimal strategies, with empirical results showing effectiveness across diverse budgets and computer vision tasks.
In the domain of Active Learning (AL), a learner actively selects which unlabeled examples to seek labels from an oracle, while operating within predefined budget constraints. Importantly, it has been recently shown that distinct query strategies are better suited for different conditions and budgetary constraints. In practice, the determination of the most appropriate AL strategy for a given situation remains an open problem. To tackle this challenge, we propose a practical derivative-based method that dynamically identifies the best strategy for a given budget. Intuitive motivation for our approach is provided by the theoretical analysis of a simplified scenario. We then introduce a method to dynamically select an AL strategy, which takes into account the unique characteristics of the problem and the available budget. Empirical results showcase the effectiveness of our approach across diverse budgets and computer vision tasks.