Graph-based Visual-Semantic Entanglement Network for Zero-shot Image Recognition
This work addresses zero-shot learning for image recognition, offering a novel method to reduce bias and ambiguity in unseen object classification, though it appears incremental as it builds on existing CNN and GCN frameworks.
The paper tackles the problem of zero-shot image recognition by addressing visual feature pattern inertia and lack of semantic relationships, proposing a Graph-based Visual-Semantic Entanglement Network that outperforms state-of-the-art methods on datasets like AwA2, CUB, and SUN with improved semantic linkage modeling.
Zero-shot learning uses semantic attributes to connect the search space of unseen objects. In recent years, although the deep convolutional network brings powerful visual modeling capabilities to the ZSL task, its visual features have severe pattern inertia and lack of representation of semantic relationships, which leads to severe bias and ambiguity. In response to this, we propose the Graph-based Visual-Semantic Entanglement Network to conduct graph modeling of visual features, which is mapped to semantic attributes by using a knowledge graph, it contains several novel designs: 1. it establishes a multi-path entangled network with the convolutional neural network (CNN) and the graph convolutional network (GCN), which input the visual features from CNN to GCN to model the implicit semantic relations, then GCN feedback the graph modeled information to CNN features; 2. it uses attribute word vectors as the target for the graph semantic modeling of GCN, which forms a self-consistent regression for graph modeling and supervise GCN to learn more personalized attribute relations; 3. it fuses and supplements the hierarchical visual-semantic features refined by graph modeling into visual embedding. Our method outperforms state-of-the-art approaches on multiple representative ZSL datasets: AwA2, CUB, and SUN by promoting the semantic linkage modelling of visual features.