Robustness in sparse artificial neural networks trained with adaptive topology
This work addresses efficiency and reliability in deep learning models for image classification, though it appears incremental as it builds on existing sparse network methods.
The paper tackles the problem of robustness in sparse neural networks by investigating adaptive topology training on MNIST and Fashion MNIST, achieving competitive accuracy with 99% sparsity while maintaining performance under perturbations like adversarial attacks and link removal.
We investigate the robustness of sparse artificial neural networks trained with adaptive topology. We focus on a simple yet effective architecture consisting of three sparse layers with 99% sparsity followed by a dense layer, applied to image classification tasks such as MNIST and Fashion MNIST. By updating the topology of the sparse layers between each epoch, we achieve competitive accuracy despite the significantly reduced number of weights. Our primary contribution is a detailed analysis of the robustness of these networks, exploring their performance under various perturbations including random link removal, adversarial attack, and link weight shuffling. Through extensive experiments, we demonstrate that adaptive topology not only enhances efficiency but also maintains robustness. This work highlights the potential of adaptive sparse networks as a promising direction for developing efficient and reliable deep learning models.