DelugeNets: Deep Networks with Efficient and Flexible Cross-layer Information Inflows
This addresses the challenge of improving information propagation and parameter efficiency in deep learning models for computer vision tasks, representing an incremental advancement over existing architectures like ResNets and DenseNets.
The paper tackles the problem of enabling efficient and flexible cross-layer information flow in deep neural networks, resulting in a DelugeNet model that achieves classification errors of 3.76% on CIFAR-10 and 19.02% on CIFAR-100 with 4.31 GigaFLOPs and 20.2M parameters, and performs competitively to ResNet-200 on ImageNet with half the computations.
Deluge Networks (DelugeNets) are deep neural networks which efficiently facilitate massive cross-layer information inflows from preceding layers to succeeding layers. The connections between layers in DelugeNets are established through cross-layer depthwise convolutional layers with learnable filters, acting as a flexible yet efficient selection mechanism. DelugeNets can propagate information across many layers with greater flexibility and utilize network parameters more effectively compared to ResNets, whilst being more efficient than DenseNets. Remarkably, a DelugeNet model with just model complexity of 4.31 GigaFLOPs and 20.2M network parameters, achieve classification errors of 3.76% and 19.02% on CIFAR-10 and CIFAR-100 dataset respectively. Moreover, DelugeNet-122 performs competitively to ResNet-200 on ImageNet dataset, despite costing merely half of the computations needed by the latter.