Achieving Domain-Independent Certified Robustness via Knowledge Continuity
This addresses the challenge of achieving domain-independent certified robustness for neural networks, which is an incremental advance over existing Lipschitz-based methods.
The paper tackles the problem of certifying neural network robustness across different input domains by introducing knowledge continuity, a novel definition that provides certification guarantees independent of domain modality, norms, and distribution, and shows it does not hinder inferential performance.
We present knowledge continuity, a novel definition inspired by Lipschitz continuity which aims to certify the robustness of neural networks across input domains (such as continuous and discrete domains in vision and language, respectively). Most existing approaches that seek to certify robustness, especially Lipschitz continuity, lie within the continuous domain with norm and distribution-dependent guarantees. In contrast, our proposed definition yields certification guarantees that depend only on the loss function and the intermediate learned metric spaces of the neural network. These bounds are independent of domain modality, norms, and distribution. We further demonstrate that the expressiveness of a model class is not at odds with its knowledge continuity. This implies that achieving robustness by maximizing knowledge continuity should not theoretically hinder inferential performance. Finally, to complement our theoretical results, we present several applications of knowledge continuity such as regularization, a certification algorithm, and show that knowledge continuity can be used to localize vulnerable components of a neural network.