An Optimization Framework for Semi-Supervised and Transfer Learning using Multiple Classifiers and Clusterers
This addresses the challenge of concept drift and distribution differences in transfer learning for machine learning practitioners, though it is incremental as it builds on existing optimization and clustering methods.
The paper tackles the problem of classifying new target data in semi-supervised and transfer learning by integrating classifier estimates from source data with cluster similarity from target data, resulting in a consensus labeling that substantially outperforms popular transductive techniques and naive classifier application.
Unsupervised models can provide supplementary soft constraints to help classify new, "target" data since similar instances in the target set are more likely to share the same class label. Such models can also help detect possible differences between training and target distributions, which is useful in applications where concept drift may take place, as in transfer learning settings. This paper describes a general optimization framework that takes as input class membership estimates from existing classifiers learnt on previously encountered "source" data, as well as a similarity matrix from a cluster ensemble operating solely on the target data to be classified, and yields a consensus labeling of the target data. This framework admits a wide range of loss functions and classification/clustering methods. It exploits properties of Bregman divergences in conjunction with Legendre duality to yield a principled and scalable approach. A variety of experiments show that the proposed framework can yield results substantially superior to those provided by popular transductive learning techniques or by naively applying classifiers learnt on the original task to the target data.