Hybrid Generative/Discriminative Learning for Automatic Image Annotation
This work addresses the challenge of automatic image annotation for applications like image retrieval, but it is incremental as it builds on existing generative and discriminative methods.
The paper tackles the problem of automatic image annotation, which involves modeling ambiguous inputs and outputs with a large tag vocabulary, by proposing a hybrid generative-discriminative classifier that addresses data ambiguity and overfitting. The result is superior annotation performance and better tag scalability, as demonstrated in experiments.
Automatic image annotation (AIA) raises tremendous challenges to machine learning as it requires modeling of data that are both ambiguous in input and output, e.g., images containing multiple objects and labeled with multiple semantic tags. Even more challenging is that the number of candidate tags is usually huge (as large as the vocabulary size) yet each image is only related to a few of them. This paper presents a hybrid generative-discriminative classifier to simultaneously address the extreme data-ambiguity and overfitting-vulnerability issues in tasks such as AIA. Particularly: (1) an Exponential-Multinomial Mixture (EMM) model is established to capture both the input and output ambiguity and in the meanwhile to encourage prediction sparsity; and (2) the prediction ability of the EMM model is explicitly maximized through discriminative learning that integrates variational inference of graphical models and the pairwise formulation of ordinal regression. Experiments show that our approach achieves both superior annotation performance and better tag scalability.