A Closer Look at Branch Classifiers of Multi-exit Architectures
This work addresses efficiency improvements for deep neural networks, but it is incremental as it focuses on analyzing existing branching patterns rather than introducing a new paradigm.
The paper tackles the problem of optimizing branch classifiers in multi-exit architectures to reduce inference time, finding that complexity-decreasing branches achieve the best accuracy-cost trade-off compared to constant or increasing complexity branches.
Multi-exit architectures consist of a backbone and branch classifiers that offer shortened inference pathways to reduce the run-time of deep neural networks. In this paper, we analyze different branching patterns that vary in their allocation of computational complexity for the branch classifiers. Constant-complexity branching keeps all branches the same, while complexity-increasing and complexity-decreasing branching place more complex branches later or earlier in the backbone respectively. Through extensive experimentation on multiple backbones and datasets, we find that complexity-decreasing branches are more effective than constant-complexity or complexity-increasing branches, which achieve the best accuracy-cost trade-off. We investigate a cause by using knowledge consistency to probe the effect of adding branches onto a backbone. Our findings show that complexity-decreasing branching yields the least disruption to the feature abstraction hierarchy of the backbone, which explains the effectiveness of the branching patterns.