Stochastic Pooling for Regularization of Deep Convolutional Neural Networks
This provides a hyper-parameter-free regularization method for deep learning practitioners, though it is incremental as it builds on existing pooling techniques.
The paper tackles the problem of regularizing large convolutional neural networks by introducing stochastic pooling, which replaces deterministic pooling with a random selection based on activation levels, achieving state-of-the-art performance on four image datasets without data augmentation.
We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined with other regularization approaches, such as dropout and data augmentation. We achieve state-of-the-art performance on four image datasets, relative to other approaches that do not utilize data augmentation.