Improving Accuracy of Nonparametric Transfer Learning via Vector Segmentation
This work addresses the challenge of enhancing transfer learning accuracy for practitioners in computationally restricted settings, though it appears incremental as it builds on existing feature extraction methods.
The paper tackled the problem of improving accuracy in nonparametric transfer learning by leveraging specific properties of deep neural network features, demonstrating that segmenting feature vectors can lead to better accuracy for distributions where information is concentrated in a few coordinates, with experiments showing improved results on vision and audio datasets.
Transfer learning using deep neural networks as feature extractors has become increasingly popular over the past few years. It allows to obtain state-of-the-art accuracy on datasets too small to train a deep neural network on its own, and it provides cutting edge descriptors that, combined with nonparametric learning methods, allow rapid and flexible deployment of performing solutions in computationally restricted settings. In this paper, we are interested in showing that the features extracted using deep neural networks have specific properties which can be used to improve accuracy of downstream nonparametric learning methods. Namely, we demonstrate that for some distributions where information is embedded in a few coordinates, segmenting feature vectors can lead to better accuracy. We show how this model can be applied to real datasets by performing experiments using three mainstream deep neural network feature extractors and four databases, in vision and audio.