The Role of Active Learning in Modern Machine Learning
This work addresses the limited practical application of Active Learning for researchers and practitioners by showing its role as a complementary tool after using more effective techniques.
The study tackled the inefficiency of Active Learning (AL) in low-data scenarios by comparing it with data augmentation (DA) and semi-supervised learning (SSL), finding that AL alone provides only 1-4% lift over random sampling, while DA and SSL can achieve up to 60% lift, but AL can still offer improvements when combined with these methods.
Even though Active Learning (AL) is widely studied, it is rarely applied in contexts outside its own scientific literature. We posit that the reason for this is AL's high computational cost coupled with the comparatively small lifts it is typically able to generate in scenarios with few labeled points. In this work we study the impact of different methods to combat this low data scenario, namely data augmentation (DA), semi-supervised learning (SSL) and AL. We find that AL is by far the least efficient method of solving the low data problem, generating a lift of only 1-4\% over random sampling, while DA and SSL methods can generate up to 60\% lift in combination with random sampling. However, when AL is combined with strong DA and SSL techniques, it surprisingly is still able to provide improvements. Based on these results, we frame AL not as a method to combat missing labels, but as the final building block to squeeze the last bits of performance out of data after appropriate DA and SSL methods as been applied.