Exploiting Nontrivial Connectivity for Automatic Speech Recognition
This work addresses performance enhancement in automatic speech recognition, but it is incremental as it applies known connectivity methods to a new domain.
The paper compared residual, densely-connected, and highway networks on image classification and applied them to automatic speech recognition, showing they provide significant improvements to existing models.
Nontrivial connectivity has allowed the training of very deep networks by addressing the problem of vanishing gradients and offering a more efficient method of reusing parameters. In this paper we make a comparison between residual networks, densely-connected networks and highway networks on an image classification task. Next, we show that these methodologies can easily be deployed into automatic speech recognition and provide significant improvements to existing models.