ParaNet - Using Dense Blocks for Early Inference
This work addresses efficiency and practicality issues in convolutional networks for researchers and practitioners, but it is incremental.
The paper tackled improving DenseNet's practicality by introducing ParaNet, a new architecture with three pipelines for early inference, and achieved competitive results on CIFAR-100.
DenseNets have been shown to be a competitive model among recent convolutional network architectures. These networks utilize Dense Blocks, which are groups of densely connected layers where the output of a hidden layer is fed in as the input of every other layer following it. In this paper, we aim to improve certain aspects of DenseNet, especially when it comes to practicality. We introduce ParaNet, a new architecture that constructs three pipelines which allow for early inference. We additionally introduce a cascading mechanism such that different pipelines are able to share parameters, as well as logit matching between the outputs of the pipelines. We separately evaluate each of the newly introduced mechanisms of ParaNet, then evaluate our proposed architecture on CIFAR-100.