Locality-Promoting Representation Learning
This work addresses a fundamental issue in CNN feature learning for researchers and practitioners, though it is incremental as it builds on existing regularization schemes.
The paper tackled the problem of learning features in convolutional neural networks by introducing Locality-promoting Regularization (LOCO-Reg), which improved accuracy across multiple architectures and datasets based on an empirical finding that filter weights are larger near the center.
This work investigates fundamental questions related to learning features in convolutional neural networks (CNN). Empirical findings across multiple architectures such as VGG, ResNet, Inception, DenseNet and MobileNet indicate that weights near the center of a filter are larger than weights on the outside. Current regularization schemes violate this principle. Thus, we introduce Locality-promoting Regularization (LOCO-Reg), which yields accuracy gains across multiple architectures and datasets. We also show theoretically that the empirical finding is a consequence of maximizing feature cohesion under the assumption of spatial locality.