Training Deeper Convolutional Networks with Deep Supervision
This addresses the problem of computational expense and training challenges in deep learning for computer vision researchers, but it is incremental as it builds on existing deep supervision techniques.
The paper tackles the difficulty of training deeper convolutional networks by adding auxiliary supervision branches at intermediate layers, which makes training easier and improves classification results on ImageNet and MIT Places datasets.
One of the most promising ways of improving the performance of deep convolutional neural networks is by increasing the number of convolutional layers. However, adding layers makes training more difficult and computationally expensive. In order to train deeper networks, we propose to add auxiliary supervision branches after certain intermediate layers during training. We formulate a simple rule of thumb to determine where these branches should be added. The resulting deeply supervised structure makes the training much easier and also produces better classification results on ImageNet and the recently released, larger MIT Places dataset