Simple and Efficient Learning using Privileged Information
This work provides an incremental improvement for researchers and practitioners in machine learning who need faster training with privileged information, such as in image categorization.
The paper tackles the problem of efficiently learning with privileged information, which is available only during training, by proposing a modified SVM+ algorithm using squared hinge loss, resulting in up to 100 times speedup with comparable accuracy on image categorization tasks.
The Support Vector Machine using Privileged Information (SVM+) has been proposed to train a classifier to utilize the additional privileged information that is only available in the training phase but not available in the test phase. In this work, we propose an efficient solution for SVM+ by simply utilizing the squared hinge loss instead of the hinge loss as in the existing SVM+ formulation, which interestingly leads to a dual form with less variables and in the same form with the dual of the standard SVM. The proposed algorithm is utilized to leverage the additional web knowledge that is only available during training for the image categorization tasks. The extensive experimental results on both Caltech101 andWebQueries datasets show that our proposed method can achieve a factor of up to hundred times speedup with the comparable accuracy when compared with the existing SVM+ method.