Generalized Zero-Shot Recognition based on Visually Semantic Embedding
This work addresses the challenge of recognizing unseen classes in zero-shot learning, which is crucial for applications like image classification with limited labeled data, but it appears incremental as it builds on prior mapping approaches with novel embeddings and models.
The paper tackles the problem of Generalized Zero-Shot Learning (GZSL) by proposing a method that is agnostic to unseen images and semantic vectors during training, using a low-dimensional visually semantic embedding to bridge the semantic gap and a graphical model for inference, resulting in significant accuracy improvements over state-of-the-art methods on benchmark datasets.
We propose a novel Generalized Zero-Shot learning (GZSL) method that is agnostic to both unseen images and unseen semantic vectors during training. Prior works in this context propose to map high-dimensional visual features to the semantic domain, we believe contributes to the semantic gap. To bridge the gap, we propose a novel low-dimensional embedding of visual instances that is "visually semantic." Analogous to semantic data that quantifies the existence of an attribute in the presented instance, components of our visual embedding quantifies existence of a prototypical part-type in the presented instance. In parallel, as a thought experiment, we quantify the impact of noisy semantic data by utilizing a novel visual oracle to visually supervise a learner. These factors, namely semantic noise, visual-semantic gap and label noise lead us to propose a new graphical model for inference with pairwise interactions between label, semantic data, and inputs. We tabulate results on a number of benchmark datasets demonstrating significant improvement in accuracy over state-of-the-art under both semantic and visual supervision.