Probing the Purview of Neural Networks via Gradient Analysis
This work addresses the challenge of detecting anomalous inputs in neural networks for applications requiring robust inference, though it appears incremental as it builds on existing gradient analysis techniques.
The paper tackles the problem of analyzing neural networks' capacity to handle inputs that differ from training data, introducing a gradient-based method to detect anomalous inputs like out-of-distribution, adversarial, and corrupted samples, achieving improvements of up to 2.7%, 19.8%, and 35.6% in AUROC scores over state-of-the-art methods.
We analyze the data-dependent capacity of neural networks and assess anomalies in inputs from the perspective of networks during inference. The notion of data-dependent capacity allows for analyzing the knowledge base of a model populated by learned features from training data. We define purview as the additional capacity necessary to characterize inference samples that differ from the training data. To probe the purview of a network, we utilize gradients to measure the amount of change required for the model to characterize the given inputs more accurately. To eliminate the dependency on ground-truth labels in generating gradients, we introduce confounding labels that are formulated by combining multiple categorical labels. We demonstrate that our gradient-based approach can effectively differentiate inputs that cannot be accurately represented with learned features. We utilize our approach in applications of detecting anomalous inputs, including out-of-distribution, adversarial, and corrupted samples. Our approach requires no hyperparameter tuning or additional data processing and outperforms state-of-the-art methods by up to 2.7%, 19.8%, and 35.6% of AUROC scores, respectively.