Dynamic Routing Networks
This addresses efficiency issues for deploying deep neural networks in real-world applications, representing an incremental improvement by customizing model capacity per instance.
The paper tackles the problem of high inference costs in deep neural networks by proposing Dynamic Routing Networks (DRNets), which enable instance-aware inference to reduce unnecessary computations, achieving substantial reductions in parameter size and FLOPs while maintaining comparable prediction performance to state-of-the-art architectures.
The deployment of deep neural networks in real-world applications is mostly restricted by their high inference costs. Extensive efforts have been made to improve the accuracy with expert-designed or algorithm-searched architectures. However, the incremental improvement is typically achieved with increasingly more expensive models that only a small portion of input instances really need. Inference with a static architecture that processes all input instances via the same transformation would thus incur unnecessary computational costs. Therefore, customizing the model capacity in an instance-aware manner is much needed for higher inference efficiency. In this paper, we propose Dynamic Routing Networks (DRNets), which support efficient instance-aware inference by routing the input instance to only necessary transformation branches selected from a candidate set of branches for each connection between transformation nodes. The branch selection is dynamically determined via the corresponding branch importance weights, which are first generated from lightweight hypernetworks (RouterNets) and then recalibrated with Gumbel-Softmax before the selection. Extensive experiments show that DRNets can reduce a substantial amount of parameter size and FLOPs during inference with prediction performance comparable to state-of-the-art architectures.