Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss
This work addresses the need for more robust neural networks against adversarial attacks, offering incremental improvements in Lipschitz-based methods.
The paper tackles the problem of improving certified robustness in deep learning by introducing a Block Reflector Orthogonal layer and a logit annealing loss function, resulting in state-of-the-art performance on datasets like CIFAR-10/100 and ImageNet.
Lipschitz neural networks are well-known for providing certified robustness in deep learning. In this paper, we present a novel, efficient Block Reflector Orthogonal (BRO) layer that enhances the capability of orthogonal layers on constructing more expressive Lipschitz neural architectures. In addition, by theoretically analyzing the nature of Lipschitz neural networks, we introduce a new loss function that employs an annealing mechanism to increase margin for most data points. This enables Lipschitz models to provide better certified robustness. By employing our BRO layer and loss function, we design BRONet - a simple yet effective Lipschitz neural network that achieves state-of-the-art certified robustness. Extensive experiments and empirical analysis on CIFAR-10/100, Tiny-ImageNet, and ImageNet validate that our method outperforms existing baselines. The implementation is available at https://github.com/ntuaislab/BRONet.