SelectScale: Mining More Patterns from Images via Selective and Soft Dropout
This work aims to improve the efficiency of pattern learning in CNNs for image recognition tasks, offering an incremental improvement over existing dropout methods.
This paper addresses the problem of convolutional neural networks (CNNs) learning only a small proportion of useful patterns from images, proposing SelectScale to selectively adjust important features instead of randomly dropping them. The method improves CNN performance on CIFAR and ImageNet datasets.
Convolutional neural networks (CNNs) have achieved remarkable success in image recognition. Although the internal patterns of the input images are effectively learned by the CNNs, these patterns only constitute a small proportion of useful patterns contained in the input images. This can be attributed to the fact that the CNNs will stop learning if the learned patterns are enough to make a correct classification. Network regularization methods like dropout and SpatialDropout can ease this problem. During training, they randomly drop the features. These dropout methods, in essence, change the patterns learned by the networks, and in turn, forces the networks to learn other patterns to make the correct classification. However, the above methods have an important drawback. Randomly dropping features is generally inefficient and can introduce unnecessary noise. To tackle this problem, we propose SelectScale. Instead of randomly dropping units, SelectScale selects the important features in networks and adjusts them during training. Using SelectScale, we improve the performance of CNNs on CIFAR and ImageNet.