Intervening in Black Box: Concept Bottleneck Model for Enhancing Human Neural Network Mutual Understanding
This work addresses the challenge of enhancing mutual understanding between humans and neural networks for researchers and practitioners dealing with black-box models, though it appears incremental as it builds on existing Concept Bottleneck Models.
The paper tackles the problem of improving interpretability and accuracy in deep learning models by proposing the Concept Bottleneck Model for Enhancing Human-Neural Network Mutual Understanding (CBM-HNMU), which identifies and refines detrimental concepts to distill corrected knowledge back into black-box models, resulting in a maximum accuracy improvement of 2.64% and a maximum increase in average accuracy of 1.03% across datasets like Flower-102 and CIFAR-10.
Recent advances in deep learning have led to increasingly complex models with deeper layers and more parameters, reducing interpretability and making their decisions harder to understand. While many methods explain black-box reasoning, most lack effective interventions or only operate at sample-level without modifying the model itself. To address this, we propose the Concept Bottleneck Model for Enhancing Human-Neural Network Mutual Understanding (CBM-HNMU). CBM-HNMU leverages the Concept Bottleneck Model (CBM) as an interpretable framework to approximate black-box reasoning and communicate conceptual understanding. Detrimental concepts are automatically identified and refined (removed/replaced) based on global gradient contributions. The modified CBM then distills corrected knowledge back into the black-box model, enhancing both interpretability and accuracy. We evaluate CBM-HNMU on various CNN and transformer-based models across Flower-102, CIFAR-10, CIFAR-100, FGVC-Aircraft, and CUB-200, achieving a maximum accuracy improvement of 2.64% and a maximum increase in average accuracy across 1.03%. Source code is available at: https://github.com/XiGuaBo/CBM-HNMU.