Hybrid multi-layer Deep CNN/Aggregator feature for image classification
This work addresses the problem of computational efficiency and data annotation burdens in image classification for researchers and practitioners, offering an incremental improvement over existing adaptation methods.
The paper tackles the high computational and data requirements of Deep Convolutional Neural Networks (DCNNs) for image classification by proposing a hybrid method that combines unsupervised aggregators like Bag-of-Words with DCNN intermediate layers, achieving performance comparable to adapted DCNNs on PASCAL VOC 2007 with a 150 times smaller feature size and reduced training and testing costs.
Deep Convolutional Neural Networks (DCNN) have established a remarkable performance benchmark in the field of image classification, displacing classical approaches based on hand-tailored aggregations of local descriptors. Yet DCNNs impose high computational burdens both at training and at testing time, and training them requires collecting and annotating large amounts of training data. Supervised adaptation methods have been proposed in the literature that partially re-learn a transferred DCNN structure from a new target dataset. Yet these require expensive bounding-box annotations and are still computationally expensive to learn. In this paper, we address these shortcomings of DCNN adaptation schemes by proposing a hybrid approach that combines conventional, unsupervised aggregators such as Bag-of-Words (BoW), with the DCNN pipeline by treating the output of intermediate layers as densely extracted local descriptors. We test a variant of our approach that uses only intermediate DCNN layers on the standard PASCAL VOC 2007 dataset and show performance significantly higher than the standard BoW model and comparable to Fisher vector aggregation but with a feature that is 150 times smaller. A second variant of our approach that includes the fully connected DCNN layers significantly outperforms Fisher vector schemes and performs comparably to DCNN approaches adapted to Pascal VOC 2007, yet at only a small fraction of the training and testing cost.