Dependency-aware Maximum Likelihood Estimation for Active Learning
This addresses a specific bottleneck in active learning for practitioners by improving model performance with fewer labeled samples, though it is incremental as it modifies an existing method.
The paper tackled the problem of sample dependencies being overlooked in active learning when using Maximum Likelihood Estimation (MLE), proposing Dependency-aware MLE (DMLE) to correct this, which achieved average accuracy improvements of 6% to 10.5% across different query batch sizes after collecting the first 100 samples.
Active learning aims to efficiently build a labeled training set by strategically selecting samples to query labels from annotators. In this sequential process, each sample acquisition influences subsequent selections, causing dependencies among samples in the labeled set. However, these dependencies are overlooked during the model parameter estimation stage when updating the model using Maximum Likelihood Estimation (MLE), a conventional method that assumes independent and identically distributed (i.i.d.) data. We propose Dependency-aware MLE (DMLE), which corrects MLE within the active learning framework by addressing sample dependencies typically neglected due to the i.i.d. assumption, ensuring consistency with active learning principles in the model parameter estimation process. This improved method achieves superior performance across multiple benchmark datasets, reaching higher performance in earlier cycles compared to conventional MLE. Specifically, we observe average accuracy improvements of 6%, 8.6%, and 10.5% for k=1, k=5, and k=10 respectively, after collecting the first 100 samples, where entropy is the acquisition function and k is the query batch size acquired at every active learning cycle. Our implementation is publicly available at: https://github.com/neu-spiral/DMLEforAL