Understanding the Benefits of Image Augmentations
This work addresses the explainability of image augmentations for researchers and practitioners in computer vision, though it is incremental in analyzing existing methods.
The study investigated how image augmentations affect different layers of ResNets using Centered Kernel Alignment, finding that deeper layers in fine-tuned models are more impacted, with multi-image augmentations having a greater effect than single-image ones.
Image Augmentations are widely used to reduce overfitting in neural networks. However, the explainability of their benefits largely remains a mystery. We study which layers of residual neural networks (ResNets) are most affected by augmentations using Centered Kernel Alignment (CKA). We do so by analyzing models of varying widths and depths, as well as whether their weights are initialized randomly or through transfer learning. We find that the pattern of how the layers are affected depends on the model's depth, and that networks trained with augmentation that use information from two images affect the learned weights significantly more than augmentations that operate on a single image. Deeper layers of ResNets initialized with ImageNet-1K weights and fine-tuned receive more impact from the augmentations than early layers. Understanding the effects of image augmentations on CNNs will have a variety of applications, such as determining how far back one needs to fine-tune a network and which layers should be frozen when implementing layer freezing algorithms.