Semantic Depth Matters: Explaining Errors of Deep Vision Networks through Perceived Class Similarities
This work addresses the need for more transparent error analysis in deep learning for researchers and practitioners, though it is incremental in building on existing evaluation methods.
The paper tackles the problem of explaining misclassifications in deep vision networks by introducing a framework that links semantic hierarchy depth to error patterns, showing that higher semantic depth improves alignment between perceived class similarities and actual errors.
Understanding deep neural network (DNN) behavior requires more than evaluating classification accuracy alone; analyzing errors and their predictability is equally crucial. Current evaluation methodologies lack transparency, particularly in explaining the underlying causes of network misclassifications. To address this, we introduce a novel framework that investigates the relationship between the semantic hierarchy depth perceived by a network and its real-data misclassification patterns. Central to our framework is the Similarity Depth (SD) metric, which quantifies the semantic hierarchy depth perceived by a network along with a method of evaluation of how closely the network's errors align with its internally perceived similarity structure. We also propose a graph-based visualization of model semantic relationships and misperceptions. A key advantage of our approach is that leveraging class templates -- representations derived from classifier layer weights -- is applicable to already trained networks without requiring additional data or experiments. Our approach reveals that deep vision networks encode specific semantic hierarchies and that high semantic depth improves the compliance between perceived class similarities and actual errors.