Assessing Systematic Weaknesses of DNNs using Counterfactuals
This addresses safety-critical applications like autonomous driving by improving testing for systematic weaknesses, though it is incremental as it builds on counterfactual explanations.
The paper tackles the problem of attributing systematic weaknesses in DNNs to specific semantic features, proposing a counterfactual-based algorithm to validate these attributions efficiently. It demonstrates on a semantic segmentation model in autonomous driving that performance differences among pedestrian assets exist, but the asset type is not always the cause.
With the advancement of DNNs into safety-critical applications, testing approaches for such models have gained more attention. A current direction is the search for and identification of systematic weaknesses that put safety assumptions based on average performance values at risk. Such weaknesses can take on the form of (semantically coherent) subsets or areas in the input space where a DNN performs systematically worse than its expected average. However, it is non-trivial to attribute the reason for such observed low performances to the specific semantic features that describe the subset. For instance, inhomogeneities within the data w.r.t. other (non-considered) attributes might distort results. However, taking into account all (available) attributes and their interaction is often computationally highly expensive. Inspired by counterfactual explanations, we propose an effective and computationally cheap algorithm to validate the semantic attribution of existing subsets, i.e., to check whether the identified attribute is likely to have caused the degraded performance. We demonstrate this approach on an example from the autonomous driving domain using highly annotated simulated data, where we show for a semantic segmentation model that (i) performance differences among the different pedestrian assets exist, but (ii) only in some cases is the asset type itself the reason for this reduction in the performance.