Fine-grain Inference on Out-of-Distribution Data with Hierarchical Classification
This addresses the need for more interpretable and informative OOD detection in machine learning systems, particularly for real-world applications where inputs may overlap with training data, though it is an incremental improvement over existing methods.
The paper tackles the problem of out-of-distribution (OOD) detection by proposing a hierarchical classification model that makes predictions at varying granularities for ambiguous inputs, improving interpretability and providing more informative outputs compared to binary anomaly detection.
Machine learning methods must be trusted to make appropriate decisions in real-world environments, even when faced with out-of-distribution (OOD) samples. Many current approaches simply aim to detect OOD examples and alert the user when an unrecognized input is given. However, when the OOD sample significantly overlaps with the training data, a binary anomaly detection is not interpretable or explainable, and provides little information to the user. We propose a new model for OOD detection that makes predictions at varying levels of granularity as the inputs become more ambiguous, the model predictions become coarser and more conservative. Consider an animal classifier that encounters an unknown bird species and a car. Both cases are OOD, but the user gains more information if the classifier recognizes that its uncertainty over the particular species is too large and predicts bird instead of detecting it as OOD. Furthermore, we diagnose the classifiers performance at each level of the hierarchy improving the explainability and interpretability of the models predictions. We demonstrate the effectiveness of hierarchical classifiers for both fine- and coarse-grained OOD tasks.