Scalable Lipschitz Estimation for CNNs
This work addresses a bottleneck in estimating Lipschitz constants for CNNs, which is important for assessing generalizability and adversarial robustness in computer vision applications, but it is incremental as it builds on existing estimation methods.
The paper tackles the limited scalability of existing Lipschitz constant estimation methods for convolutional neural networks (CNNs) by proposing a novel partitioning approach that accelerates estimation, demonstrating enhanced scalability and comparable accuracy in experiments.
Estimating the Lipschitz constant of deep neural networks is of growing interest as it is useful for informing on generalisability and adversarial robustness. Convolutional neural networks (CNNs) in particular, underpin much of the recent success in computer vision related applications. However, although existing methods for estimating the Lipschitz constant can be tight, they have limited scalability when applied to CNNs. To tackle this, we propose a novel method to accelerate Lipschitz constant estimation for CNNs. The core idea is to divide a large convolutional block via a joint layer and width-wise partition, into a collection of smaller blocks. We prove an upper-bound on the Lipschitz constant of the larger block in terms of the Lipschitz constants of the smaller blocks. Through varying the partition factor, the resulting method can be adjusted to prioritise either accuracy or scalability and permits parallelisation. We demonstrate an enhanced scalability and comparable accuracy to existing baselines through a range of experiments.