Learning how to Active Learn: A Deep Reinforcement Learning Approach
This addresses the limitation of heuristic selection methods in active learning, which vary in effectiveness across datasets, by providing a learned approach that can transfer across languages.
The paper tackles the problem of selecting data for annotation in active learning by reframing it as a reinforcement learning task to learn a selection policy, resulting in uniform improvements over traditional heuristic methods in cross-lingual named entity recognition.
Active learning aims to select a small subset of data for annotation such that a classifier learned on the data is highly accurate. This is usually done using heuristic selection methods, however the effectiveness of such methods is limited and moreover, the performance of heuristics varies between datasets. To address these shortcomings, we introduce a novel formulation by reframing the active learning as a reinforcement learning problem and explicitly learning a data selection policy, where the policy takes the role of the active learning heuristic. Importantly, our method allows the selection policy learned using simulation on one language to be transferred to other languages. We demonstrate our method using cross-lingual named entity recognition, observing uniform improvements over traditional active learning.