Mapping DNN Embedding Manifolds for Network Generalization Prediction
This addresses the challenge of deploying DNNs in safety-critical, unconstrained environments such as self-driving vehicles and medical imaging, offering a novel approach for generalization prediction.
The paper tackles the problem of predicting how deep neural networks (DNNs) will generalize to new operating domains without requiring labeled data or domain knowledge, achieving state-of-the-art results in 13 out of 15 tasks for applications like pedestrian and melanoma classification.
Understanding Deep Neural Network (DNN) performance in changing conditions is essential for deploying DNNs in safety critical applications with unconstrained environments, e.g., perception for self-driving vehicles or medical image analysis. Recently, the task of Network Generalization Prediction (NGP) has been proposed to predict how a DNN will generalize in a new operating domain. Previous NGP approaches have relied on labeled metadata and known distributions for the new operating domains. In this study, we propose the first NGP approach that predicts DNN performance based solely on how unlabeled images from an external operating domain map in the DNN embedding space. We demonstrate this technique for pedestrian, melanoma, and animal classification tasks and show state of the art NGP in 13 of 15 NGP tasks without requiring domain knowledge. Additionally, we show that our NGP embedding maps can be used to identify misclassified images when the DNN performance is poor.