A Closer Look at the Learnability of Out-of-Distribution (OOD) Detection
This work addresses the problem of improving theoretical understanding for researchers in machine learning, offering incremental insights by applying PAC learning theory to OOD detection.
The paper tackles the theoretical learnability of out-of-distribution (OOD) detection by distinguishing between uniform and non-uniform learnability, showing that non-uniform learnability can turn negative results into positive ones in several cases, and provides concrete learning algorithms with sample-complexity analysis where learnable.
Machine learning algorithms often encounter different or "out-of-distribution" (OOD) data at deployment time, and OOD detection is frequently employed to detect these examples. While it works reasonably well in practice, existing theoretical results on OOD detection are highly pessimistic. In this work, we take a closer look at this problem, and make a distinction between uniform and non-uniform learnability, following PAC learning theory. We characterize under what conditions OOD detection is uniformly and non-uniformly learnable, and we show that in several cases, non-uniform learnability turns a number of negative results into positive. In all cases where OOD detection is learnable, we provide concrete learning algorithms and a sample-complexity analysis.