Improving Out-of-Distribution Detection via Epistemic Uncertainty Adversarial Training
This addresses a critical issue for deploying machine learning in safety-critical applications by enhancing OOD detection performance under real-world constraints.
The paper tackled the problem of unreliable out-of-distribution (OOD) detection at low false positive rates (≤1%) by proposing an adversarial training scheme that attacks epistemic uncertainty in dropout ensembles, improving standardized partial AUC from near-random guessing to ≥0.75.
The quantification of uncertainty is important for the adoption of machine learning, especially to reject out-of-distribution (OOD) data back to human experts for review. Yet progress has been slow, as a balance must be struck between computational efficiency and the quality of uncertainty estimates. For this reason many use deep ensembles of neural networks or Monte Carlo dropout for reasonable uncertainty estimates at relatively minimal compute and memory. Surprisingly, when we focus on the real-world applicable constraint of $\leq 1\%$ false positive rate (FPR), prior methods fail to reliably detect OOD samples as such. Notably, even Gaussian random noise fails to trigger these popular OOD techniques. We help to alleviate this problem by devising a simple adversarial training scheme that incorporates an attack of the epistemic uncertainty predicted by the dropout ensemble. We demonstrate this method improves OOD detection performance on standard data (i.e., not adversarially crafted), and improves the standardized partial AUC from near-random guessing performance to $\geq 0.75$.