Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?
This provides a theoretical foundation and practical algorithm for active learning in regression tasks like human pose estimation, though it's an incremental extension of existing methods.
The paper tackles the problem of active learning for regression by theoretically deriving the Expected Gradient Length formulation and showing its equivalence to Bayesian uncertainty, then proposes EGL++ algorithm that approximates ensemble effects with a single network and extends the approach to human pose estimation, achieving competitive performance on MPII and LSP/LSPET datasets.
Active learning algorithms select a subset of data for annotation to maximize the model performance on a budget. One such algorithm is Expected Gradient Length, which as the name suggests uses the approximate gradient induced per example in the sampling process. While Expected Gradient Length has been successfully used for classification and regression, the formulation for regression remains intuitively driven. Hence, our theoretical contribution involves deriving this formulation, thereby supporting the experimental evidence. Subsequently, we show that expected gradient length in regression is equivalent to Bayesian uncertainty. If certain assumptions are infeasible, our algorithmic contribution (EGL++) approximates the effect of ensembles with a single deterministic network. Instead of computing multiple possible inferences per input, we leverage previously annotated samples to quantify the probability of previous labels being the true label. Such an approach allows us to extend expected gradient length to a new task: human pose estimation. We perform experimental validation on two human pose datasets (MPII and LSP/LSPET), highlighting the interpretability and competitiveness of EGL++ with different active learning algorithms for human pose estimation.