Repairing Group-Level Errors for DNNs Using Weighted Regularization
This work addresses critical reliability issues in DNNs used in high-stakes applications, offering a novel approach to fix group-level errors, though it is incremental as it builds on prior detection methods.
The authors tackled the problem of group-level errors in deep neural networks, such as confusion and bias errors, by proposing Weighted Regularization (WR) methods that significantly reduce these errors with minimal impact on overall performance across six datasets and architectures.
Deep Neural Networks (DNNs) have been widely used in software making decisions impacting people's lives. However, they have been found to exhibit severe erroneous behaviors that may lead to unfortunate outcomes. Previous work shows that such misbehaviors often occur due to class property violations rather than errors on a single image. Although methods for detecting such errors have been proposed, fixing them has not been studied so far. Here, we propose a generic method called Weighted Regularization (WR) consisting of five concrete methods targeting the error-producing classes to fix the DNNs. In particular, it can repair confusion error and bias error of DNN models for both single-label and multi-label image classifications. A confusion error happens when a given DNN model tends to confuse between two classes. Each method in WR assigns more weights at a stage of DNN retraining or inference to mitigate the confusion between target pair. A bias error can be fixed similarly. We evaluate and compare the proposed methods along with baselines on six widely-used datasets and architecture combinations. The results suggest that WR methods have different trade-offs but under each setting at least one WR method can greatly reduce confusion/bias errors at a very limited cost of the overall performance.