Guided Dropout
This work addresses overfitting issues for deep learning practitioners, offering an incremental improvement over traditional dropout methods.
The paper tackled the problem of overfitting in deep neural networks by proposing guided dropout, which selects nodes for dropout based on their strength rather than randomly, and demonstrated improved generalization across multiple datasets including MNIST, CIFAR10, CIFAR100, SVHN, and Tiny ImageNet.
Dropout is often used in deep neural networks to prevent over-fitting. Conventionally, dropout training invokes \textit{random drop} of nodes from the hidden layers of a Neural Network. It is our hypothesis that a guided selection of nodes for intelligent dropout can lead to better generalization as compared to the traditional dropout. In this research, we propose "guided dropout" for training deep neural network which drop nodes by measuring the strength of each node. We also demonstrate that conventional dropout is a specific case of the proposed guided dropout. Experimental evaluation on multiple datasets including MNIST, CIFAR10, CIFAR100, SVHN, and Tiny ImageNet demonstrate the efficacy of the proposed guided dropout.