IALE: Imitating Active Learner Ensembles
This addresses the challenge of adapting AL heuristics to specific classifier and data structures, offering a domain-specific improvement for machine learning practitioners.
The paper tackles the problem of active learning (AL) heuristic performance variability by proposing an imitation learning scheme that selects the best expert heuristic at each AL stage, resulting in outperforming state-of-the-art imitation learners and heuristics on well-known datasets.
Active learning (AL) prioritizes the labeling of the most informative data samples. However, the performance of AL heuristics depends on the structure of the underlying classifier model and the data. We propose an imitation learning scheme that imitates the selection of the best expert heuristic at each stage of the AL cycle in a batch-mode pool-based setting. We use DAGGER to train the policy on a dataset and later apply it to datasets from similar domains. With multiple AL heuristics as experts, the policy is able to reflect the choices of the best AL heuristics given the current state of the AL process. Our experiment on well-known datasets show that we both outperform state of the art imitation learners and heuristics.