Minimally Supervised Feature Selection for Classification (Master's Thesis, University Politehnica of Bucharest)
This work addresses the challenge of efficient feature selection for classification tasks, particularly in scenarios with limited labeled data, though it appears incremental as it builds on existing supervised approaches.
The paper tackles the problem of feature selection in high-dimensional data by proposing a minimally supervised algorithm that requires as few as one labeled sample per class and can handle many unlabeled samples. The method outperforms established techniques like AdaBoost and SVM in speed and accuracy, with constant training time and significant performance gains on datasets such as YouTube-Objects and MNIST.
In the context of the highly increasing number of features that are available nowadays we design a robust and fast method for feature selection. The method tries to select the most representative features that are independent from each other, but are strong together. We propose an algorithm that requires very limited labeled data (as few as one labeled frame per class) and can accommodate as many unlabeled samples. We also present here the supervised approach from which we started. We compare our two formulations with established methods like AdaBoost, SVM, Lasso, Elastic Net and FoBa and show that our method is much faster and it has constant training time. Moreover, the unsupervised approach outperforms all the methods with which we compared and the difference might be quite prominent. The supervised approach is in most cases better than the other methods, especially when the number of training shots is very limited. All that the algorithm needs is to choose from a pool of positively correlated features. The methods are evaluated on the Youtube-Objects dataset of videos and on MNIST digits dataset, while at training time we also used features obtained on CIFAR10 dataset and others pre-trained on ImageNet dataset. Thereby, we also proved that transfer learning is useful, even though the datasets differ very much: from low-resolution centered images from 10 classes, to high-resolution images with objects from 1000 classes occurring in different regions of the images or to very difficult videos with very high intraclass variance. 7